R12.0.X APPS Clone 3% 和 R12.1.X 0% 过不去,等待很长时间然后报错:RC-50004

现象如下:

Clone APPS tier时,执行了adcfgclone.pl后停在0%(12.0.X是停在3%)过不去,等待很长时间后(1小时左右),报错如下:

/ 0% completed RC-50004: Fatal: Error occurred in ApplyAppsTechStack:
RC-50013: Fatal: Failed to instantiate driver /apps/compard/apps/tech_st/10.1.2/appsutil/driver/regclone.drv

ERROR while running Apply...
Thu Sep 9 14:24:21 2010

ERROR: Failed to execute /apps/compard/apps/apps_st/comn/clone/bin/adclone.pl

然后退出,查看日志$INST_TOP/admin/log/Apply*.log,报错如下:

script returned:
****************************************************


Timed out( 3750000 ): Interrupted Exception
Beginning OUI CLI cloning for s_tools_ohThu Sep 9 13:20:31 2010
/apps/compard/apps/tech_st/10.1.3/jdk/bin/java -classpath /apps/compard/apps/apps_st/comn/clone/jlib/java:/apps/compard/apps/tech_st/10.1.2/oui/jlib/OraInstaller.jar:/apps/compard/apps/apps_st/comn/clone/jlib/xmlparserv2.jar:/apps/compard/apps/apps_st/comn/clone/jlib/ojdbc14.jar oracle.apps.ad.clone.util.OracleHomeCloner -OUICLI -e /apps/compard/inst/apps/COMPARD_sunt514001/appl/admin/TEST_sunt514001.xml -nolink -oaVar s_tools_oh -homestub tools -log /apps/compard/inst/apps/COMPARD_sunt514001/admin/log/ohclone.log

...

 

AutoConfig could not successfully execute the following scripts:
Directory: /apps/compard/apps/tech_st/10.1.3/perl/bin/perl -I /apps/compard/apps/tech_st/10.1.3/perl/lib/5.8.3 -I /apps/compard/apps/tech_st/10.1.3/perl/lib/site_perl/5.8.3 -I /apps/compard/apps/apps_st/appl/au/12.0.0/perl -I /apps/compard/apps/tech_st/10.1.3/Apache/Apache/mod_perl/lib/site_perl/5.8.3/sun4-solaris-thread-multi /apps/compard/apps/tech_st/10.1.2/appsutil/clone
ouicli.pl INSTE8_APPLY -1


AutoConfig is exiting with status 1

 

再看/apps/compard/inst/apps/COMPARD_sunt514001/admin/log/ohclone.log,是在执行10.1.2 Clone时报的错。

 

处理过程:

猜想还是Global Inventory的问题,先是建立一个新目录/home/compard/oraInventory,并赋予0777的权限,在/var/opt/oracle/下建立oraInst.loc文件,内容为inventory_loc=/home/compard/oraInventory,继续Clone,出错信息变为:

 

Executing script in InstantiateFile:
/apps/compard/apps/tech_st/10.1.3/perl/bin/perl -I /apps/compard/apps/tech_st/10.1.3/perl/lib/5.8.3 -
I /apps/compard/apps/tech_st/10.1.3/perl/lib/site_perl/5.8.3 -I /apps/compard/apps/apps_st/appl/au/12
.0.0/perl -I /apps/compard/apps/tech_st/10.1.3/Apache/Apache/mod_perl/lib/site_perl/5.8.3/sun4-sol
aris-thread-multi /apps/compard/apps/tech_st/10.1.2/appsutil/clone/ouicli.pl

script returned:
****************************************************
Timed out( 3750000 ): Interrupted Exception
Beginning OUI CLI cloning for s_tools_ohFri Sep 10 11:21:50 2010
/apps/compard/apps/tech_st/10.1.3/jdk/bin/java -classpath /apps/compard/apps/apps_st/comn/clone/jlib/
java:/apps/compard/apps/tech_st/10.1.2/oui/jlib/OraInstaller.jar:/apps/compard/apps/apps_st/comn/clon
e/jlib/xmlparserv2.jar:/apps/compard/apps/apps_st/comn/clone/jlib/ojdbc14.jar  oracle.apps.ad.clon
e.util.OracleHomeCloner  -OUICLI -e /apps/compard/inst/apps/COMPARD_sunt514001/appl/admin/TEST_sunt51
4001.xml -nolink -oaVar s_tools_oh -homestub tools -log /apps/compard/inst/apps/COMPARD_sunt514001/ad
min/log/ohclone.log
NON-COMPLIANT: /apps/compard/apps/tech_st/10.1.2/oraInst.loc does not point to an inventory inside
 the current ORACLE_HOME
Rapid Clone only supports oraInst.loc at that location if its content points to an inventory in
side the same ORACLE_HOME
Please make the necessary changes to the following file:
/apps/compard/apps/tech_st/10.1.2/oraInst.loc
Running OUI CLI home cloning from within OracleHomeCloner:
  /apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller -debug -clone -silent -force -nolink -wai
tForCompletion -invPtrLoc /var/opt/oracle/oraInst.loc session:ORACLE_HOME=/apps/compard/apps/tech_
st/10.1.2 oracle.as.j2ee.top:s_asInstanceName=TEST_TOOLS__apps_compard_apps_tech_st_10_1_2 oracle.
as.j2ee.top:s_adminName=ias_admin oracle.as.j2ee.top:s_adminPassword=welcome ORACLE_HOME_NAME=T
EST_TOOLS__apps_compard_apps_tech_st_10_1_2 -J-Doracle.installer.noLink=true
****************************************************

看来还不能放在/var/下,我又将/var/opt/oracle下面文件删除,在10.1.2/10.1.3下作如下动作:

mkdir /apps/compard/apps/tech_st/10.1.2/oraInventory
mkdir /apps/compard/apps/tech_st/10.1.3/oraInventory
echo  inventory_loc=/apps/compard/apps/tech_st/10.1.2/oraInventory >/apps/compard/apps/tech_st/10.1.2/oraInst.loc
echo  inventory_loc=/apps/compard/apps/tech_st/10.1.3/oraInventory >/apps/compard/apps/tech_st/10.1.3/oraInst.loc

 

问题并没有得到解决,继续出错:


****************************************************
Timed out( 3750000 ): Interrupted Exception
Beginning OUI CLI cloning for s_tools_ohFri Sep 10 12:54:07 2010
/apps/compard/apps/tech_st/10.1.3/jdk/bin/java -classpath /apps/compard/apps/apps_st/comn/clone/jlib/
java:/apps/compard/apps/tech_st/10.1.2/oui/jlib/OraInstaller.jar:/apps/compard/apps/apps_st/comn/clon
e/jlib/xmlparserv2.jar:/apps/compard/apps/apps_st/comn/clone/jlib/ojdbc14.jar  oracle.apps.ad.clon
e.util.OracleHomeCloner  -OUICLI -e /apps/compard/inst/apps/COMPARD_sunt514001/appl/admin/COMPARD_sunt51
4001.xml -nolink -oaVar s_tools_oh -homestub tools -log /apps/compard/inst/apps/COMPARD_sunt514001/ad
min/log/ohclone.log
Running OUI CLI home cloning from within OracleHomeCloner:
  /apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller -debug -clone -silent -force -nolink -wai
tForCompletion -invPtrLoc /apps/compard/apps/tech_st/10.1.2/oraInst.loc session:ORACLE_HOME=/apps/
compard/apps/tech_st/10.1.2 oracle.as.j2ee.top:s_asInstanceName=TEST_TOOLS__apps_compard_apps_tech_st
_10_1_2 oracle.as.j2ee.top:s_adminName=ias_admin oracle.as.j2ee.top:s_adminPassword=welcome ORA
CLE_HOME_NAME=TEST_TOOLS__apps_compard_apps_tech_st_10_1_2 -J-Doracle.installer.noLink=true

 

这只能说与inventory的位置没什么关系,尝试改改环境变量看看:

mv /apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller /apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller.bbb

建立新的runInstaller,内容如下:

ORACLE_HOME=/apps/compard/apps/tech_st/10.1.2

export ORACLE_HOME

PATH=$ORACLE_HOME/bin:$PATH

export PATH

 

 /apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller.bbb $*

 

问题依旧,恢复原来的runInstaller,再尝试:

export TMP=/tmp

####以下指令为1行:######

/apps/compard/apps/tech_st/10.1.3/perl/bin/perl -I /apps/compard/apps/tech_st/10.1.3/perl/lib/5.8.3 -
I /apps/compard/apps/tech_st/10.1.3/perl/lib/site_perl/5.8.3 -I /apps/compard/apps/apps_st/appl/au/12
.0.0/perl -I /apps/compard/apps/tech_st/10.1.3/Apache/Apache/mod_perl/lib/site_perl/5.8.3/sun4-sol
aris-thread-multi /apps/compard/apps/tech_st/10.1.2/appsutil/clone/ouicli.pl

执行了1个多小时,没有任何反应,等啊等啊,等待了2个多小时,终于执行完毕了,检查了一下oraInventory里面的ContentsXML/inventory.xml,10.1.2已经注册上去。看来指令是没有问题的,应该是执行的太长导致的。

 

原因找到了,但是要找到解决方法,还得继续研究

...

 

先看看在这1个小时内Hang的时候,看看Clone动作在哪一步:

 

============================ohclone.log==========================
copying /apps/compard/apps/tech_st/10.1.2/jlib/ojmisc.jar into jdk/jre/lib
checking - /apps/compard/apps/tech_st/10.1.2/oraInst.loc
Validating oraInst.loc - /apps/compard/apps/tech_st/10.1.2/oraInst.loc
Found oraInst.loc at - /apps/compard/apps/tech_st/10.1.2/oraInst.loc
removing files from - /apps/compard/apps/tech_st/10.1.2/oraInventory
Checking for 11g oracle home
Running OUI CLI home cloning from within OracleHomeCloner:
  /apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller -debug -clone -silent -force -nolink -waitForCompletion -invPtrLoc /apps/compard/apps/tech_st/10.1.2/oraInst.loc session:ORACLE_HOME=/apps/compard/apps/tech_st/10.1.2 oracle.as.j2ee.top:s_asInstanceName=TEST_TOOLS__apps_compard_apps_tech_st_10_1_2 oracle.as.j2ee.top:s_adminName=ias_admin oracle.as.j2ee.top:s_adminPassword=welcome ORACLE_HOME_NAME=TEST_TOOLS__apps_compard_apps_tech_st_10_1_2 -J-Doracle.installer.noLink=true
******************************
Executing command sh -c "/apps/compard/apps/tech_st/10.1.2/oui/bin/runInstaller -debug -clone -silent -force -nolink -waitForCompletion -invPtrLoc /apps/compard/apps/tech_st/10.1.2/oraInst.loc session:ORACLE_HOME=/apps/compard/apps/tech_st/10.1.2 oracle.as.j2ee.top:s_asInstanceName=TEST_TOOLS__apps_test_apps_tech_st_10_1_2 oracle.as.j2ee.top:s_adminName=ias_admin oracle.as.j2ee.top:s_adminPassword=welcome ORACLE_HOME_NAME=TEST_TOOLS__apps_compard_apps_tech_st_10_1_2 -J-Doracle.installer.noLink=true ; echo ERRORCODE = $? ERRORCODE_END" Sat Sep 11 08:59:25 CST 2010

==========================ApplyAppsTier_09110853.log===========
instantiate file:
   source : /apps/compard/apps/tech_st/10.1.2/appsutil/template/ftrace_cfg_1012.tmp
   dest   : /apps/compard/apps/tech_st/10.1.2/forms/server/ftrace.cfg
   backup : /apps/compard/apps/tech_st/10.1.2/forms/server/ftrace.cfg to /apps/compard/inst/apps/COMPARD_sunt514001/out/ftrace.cfg
   setting permissions: 700
   setting ownership: appltest:dba

Executing script in InstantiateFile:
/apps/compard/apps/tech_st/10.1.3/perl/bin/perl -I /apps/compard/apps/tech_st/10.1.3/perl/lib/5.8.3 -I /apps/compard/apps/tech_st/10.1.3/perl/lib/site_perl/5.8.3 -I /apps/compard/apps/apps_st/appl/au/12.0.0/perl -I /apps/compard/apps/tech_st/10.1.3/Apache/Apache/mod_perl/lib/site_perl/5.8.3/sun4-solaris-thread-multi /apps/compard/apps/tech_st/10.1.2/appsutil/clone/ouicli.pl


===========================10.1.2/inventory/logs,最新的日志================

variableName = InfrastructureUse
Query Returned: false
Setting variable 'b_InfrastructureUse' to 'false'. Received the value from a code block.
Setting variable 'iASinstancePW' to ''. Received the value from a code block.
Setting variable 'b_ShowInstanceDialog' to 'false'. Received the value from a code block.
Calling Query SpawnLongQueries1.1.5  LongSpawn

commandList = /apps/compard/apps/tech_st/10.1.2/perl/bin/perl,-e,open(FILE,">/apps/compard/apps/tech_st/10.1.2/bin/getopmnproc.pl");print
FILE "#!//usr//bin//perl/nopen (LOG1, /">/apps/compard/apps/tech_st/10.1.2//bin//opmn_status.log/");/$statusout = /`//apps/compard/apps/te
ch_st/10.1.2//opmn//bin//opmnctl status /`;print LOG1 /$statusout; close LOG1;open (LOG2, /'>/apps/compard/apps/tech_st/10.1.2//bin//op
mnproc/');print LOG2 /'opmnProcName=/';open (LOG1, /'</apps/compard/apps/tech_st/10.1.2//bin//opmn_status.log/');while (<LOG1>){if (/$_
 =~ //Alive//g){chomp(/$_);/$procname=(split/" /",/$_)[2];print LOG2 /"/$procname,/";}} close LOG1;close LOG2;";exit 0;,
Query Returned: 0
Setting variable 'nReturn' to '0'. Received the value from a code block.
Calling Query SpawnLongQueries1.1.5  LongSpawn

commandList = /apps/compard/apps/tech_st/10.1.2/perl/bin/perl,-e,chmod (0755, "/apps/compard/apps/tech_st/10.1.2/bin/getopmnproc.pl");exit
 0;,
Query Returned: 0
Setting variable 'nReturn' to '0'. Received the value from a code block.
Calling Query SpawnLongQueries1.1.5  LongSpawn

commandList = /apps/compard/apps/tech_st/10.1.2/perl/bin/perl,/apps/compard/apps/tech_st/10.1.2/bin/getopmnproc.pl,

 

继续,根据官方的一些提示,重新做一次Clone,检查10.1.2/inventory/logs日志,当发现日志不再增长时,使用ps -ef|grep opmn检查OS上的opmn进程,kill之,察看监控10.1.2/inventory/logs/Clone*.log,发现继续日志在增长,看来是getopmnproc.pl hang这个指令Hang住了,继续监控日志,发现执行下列指令时,又hang住了:

platGroup = UNIX
Query Returned: true
Calling Action SpawnActions10.1.0.3.2  Spawn
        installcommand = /apps/compard/apps/tech_st/10.1.2/opmn/bin/opmnctl start
        deinstallcommand =
        WaitForCompletion = true

kill之,日志又继续增长,继续监控日志,发现执行下列指令时,又一次hang住了:

 

Start output from spawned process:
----------------------------------

Calling Action SpawnActions10.1.0.3.2  Spawn
        installcommand =
        deinstallcommand = /apps/compard/apps/tech_st/10.1.2/opmn/bin/opmnctl stopall
        WaitForCompletion = true

Calling Query RepositoryQueries3.0.0.2.26  GetTopLevelComp

 

kill之,等了几分钟后,日志不再增长,察看内容,已经出现了注册成功的信息。Clone的进度已经跳到50%了,终于可以松一口气了...

 

再次回过头来查看kill process的过程,不难发现凡是hang住的地方基本上都是与opmn有关系。然后回到操作系统上执行一下下列指令验证一下:

/apps/compard/apps/tech_st/10.1.2/opmn/bin/opmnctl start
果真这条指令执行了半个小时还没结果。最后我把这些信息发给官方的人员确认,他们回复是:clone的脚本所调用的一些脚本中,部分脚本包含了OPMN打开关闭的信息,但其实Clone的时候,根本不需要打开关闭OPMN,由于这些脚本不方便去修改,所以他们都是建议让客户手工从OS删除OPMN进程,官方的方案中也是这样写的,但是有个问题,如果你不研究日志的话,kill OPMN的时间点不好拿捏,过早或者过晚都不能解决问题。
既然OPMN不需要,那是否可以在启动它的脚本OPMNCTL上下功夫呢?
呵呵,根据这个思路,作了几个方案,试了一下,结果证明这个思路是可以让Clone正常结束,不再出现0%/3%的问题。

 

最后我选择了一个比较安全的方案提交到metalink上。一天后,接到Oracle的global service team打电话,方案在审核过程中,又过几天电话通知我,说他们已经将我的方案写成了一个Note 1207074.1,不过还在审核中,看来这几天辛苦的研究没有白费。由于这个方案风险未知,就暂不说明了。请关注Metalink,如果这个note被发布出来,那就说明我的方案确实可行。

阅读更多
个人分类: OracleERP_BI
想对作者说点什么? 我来说一句

没有更多推荐了,返回首页

关闭
关闭
关闭