RDBMS 11.2.0.4
节点1无法启动,节点2 启动正常。
节点1的集群状态一直这样
[grid@ractest1 admin]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager
但是节点2的集群状态是正常的。重启节点1和节点2 ,问题依旧。
在关闭节点1的集群的时候,提示没有集群,加参数 -f后强制可以关闭。
查看节点1的相关log :
2022-06-24 00:05:30.584:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:05:35.813:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:05:41.031:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:05:41.244:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:05:46.470:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:05:51.645:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:05:56.871:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:06:02.084:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:06:02.294:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
2022-06-24 00:06:07.516:
[/u01/app/grid/bin/oraagent.bin(23516)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/grid/log/ractest1/agent/ohasd/oraagent_grid/oraagent_grid.log"
继续查看相关log
-- asm log
Fatal NI connect error 12649, connecting to:
(LOCAL=NO)
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.4.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production
Time: 01-JUN-2022 14:38:25
Tracing not turned on.
Tns error struct:
ns main err code: 12649
TNS-12649: Unknown encryption or data integrity algorithm
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
opiodr aborting process unknown ospid (47070) as a result of ORA-609
Wed Jun 01 14:43:25 2022
***********************************************************************
2022-06-24 00:06:23.291: [ora.asm][532666112]{0:0:2} [start] InstAgent::start call sysresv
2022-06-24 00:06:23.291: [ora.asm][532666112]{0:0:2} [start] Container:start scls_clean_oracle_ipc Container +ASM1 dbHome /u01/app/grid
2022-06-24 00:06:23.301: [ora.asm][532666112]{0:0:2} [start] clsnInstConnection::makeConnectStr UsrOraEnv m_oracleHome /u01/app/grid Crshome /u01/app/grid
2022-06-24 00:06:23.301: [ora.asm][532666112]{0:0:2} [start] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/u01/app/grid/bin/oracle)(ARGV0=oracle+ASM1)(ENVS='ORACLE_HOME=/u01/app/grid,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM1)))
2022-06-24 00:06:23.302: [ora.asm][532666112]{0:0:2} [start] InstConnection::connectInt: server not attached
2022-06-24 00:06:23.314: [ora.asm][532666112]{0:0:2} [start] ORA-12547: TNS:lost contact
2022-06-24 00:06:23.314: [ora.asm][532666112]{0:0:2} [start] InstConnection::connectInt (1) Exception OCIException
2022-06-24 00:06:23.314: [ora.asm][532666112]{0:0:2} [start] InstConnection:connect:excp OCIException OCI error 12547
2022-06-24 00:06:23.314: [ora.asm][532666112]{0:0:2} [start] InstConnection::connectInt: server not attached
2022-06-24 00:06:23.323: [ora.asm][532666112]{0:0:2} [start] ORA-12547: TNS:lost contact
尝试启动asm实例,报错
[grid@ractest1 ~]$ sqlplus /nolog
SQL*Plus: Release 11.2.0.4.0 Production on Fri Jun 24 00:30:09 2022
Copyright (c) 1982, 2013, Oracle. All rights reserved.
SQL> conn / as sysasm
ERROR:
ORA-12547: TNS:lost contact
SQL> exit
MOS上搜文档,查到相关文档:11gR2 ASM Fails to Start With "ORA-12547: TNS:lost contact" and ORA-12649 due to Incorrect Network Security Setting (Doc ID 1505025.1)
与该文档中的相关日志进行对比,一抹一样。
该文档定位为grid用户下的sqlnet.ora中设置了加密。查看相关内容,果然加密
[grid@ractest1 admin]$ more sqlnet.ora
# sqlnet.ora.ractest1 Network Configuration File: /u01/app/grid/network/admin/sqlnet.ora.ractest1
# Generated by Oracle configuration tools.
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
ADR_BASE = /u01/app/gridbase
#SQLNET.ENCRYPTION_SERVER = REQUIRED
#SQLNET.ENCRYPTION_TYPES_SERVER = RC4_256
#SQLNET.CRYPTO_CHECKSUM_SERVER = REQUIRED
#SQLNET.CRYPTO_CHECKSUM_TYPES_SERVER = RC4_256
[grid@ractest1 admin]$
另一个节点就没有加密:
[grid@ractest2 admin]$ more sqlnet.ora
# sqlnet.ora.ractest2 Network Configuration File: /u01/app/grid/network/admin/sqlnet.ora.ractest2
# Generated by Oracle configuration tools.
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
ADR_BASE = /u01/app/gridbase
[grid@ractest2 admin]$
根据mos上的说明,屏蔽掉加密,再次启动,正常启动。问题解决。
参考文档:11gR2 ASM Fails to Start With "ORA-12547: TNS:lost contact" and ORA-12649 due to Incorrect Network Security Setting (Doc ID 1505025.1)
文档内容:
APPLIES TO:
Oracle Database - Enterprise Edition - Version 11.2.0.1 and later
Information in this document applies to any platform.
SYMPTOMS
11gR2 ASM fails to start while GI is restarting:
- Output from "crsctl start res ora.asm -init" command
CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
ORA-12547: TNS:lost contact
CRS-5017: The resource action "ora.asm start" encountered the following error:
ORA-12547: TNS:lost contactCRS-2674: Start of 'ora.asm' on 'racnode1' failed
CRS-2679: Attempting to clean 'ora.asm' on 'racnode1'
ORA-12547: TNS:lost contact
CRS-2681: Clean of 'ora.asm' on 'racnode1' succeeded
CRS-4000: Command Start failed, or completed with errors.
- Output from sqlplus
$ export ORACLE_SID=<+ASMn>
$ export ORACLE_HOME=<GRID_HOME>
$ $ORACLE_HOME/bin/sqlplus / as sysasm
SQL*Plus: Release 11.2.0.2.0 Production on Fri Nov 9 16:38:52 2012
Copyright (c) 1982, 2010, Oracle. All rights reserved.
ERROR:
ORA-12547: TNS:lost contact
- <GRID_USER_HOME_PATH>/oradiag_<GRID_USER>/diag/clients/user_<GRID_USER>/host_<HOST_ID>/trace/sqlnet.log
Fatal NI connect error 12547, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/orcl/grid/oracle/product/11.2.0.2/grid/bin/oracle)(ARGV0=oracle+ASM1)(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))')(DETACH=NO))(CONNECT_DATA=(CID=(PROGRAM=sqlplus)(HOST=racnode1)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.2.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.2.0 - Production
Time: 09-NOV-2012 17:36:36
Tracing not turned on.
Tns error struct:
ns main err code: 12547
TNS-12547: TNS:lost contact
ns secondary err code: 12560
nt main err code: 517
TNS-00517: Lost contact
nt secondary err code: 32
nt OS err code: 0
- <GRID_HOME>/rdbms/log/+asm1_ora_<PID>.trc
*** 2012-11-09 18:12:51.661
opiino: Attach failed due to ORA-12649
CAUSE
sqlnet.ora changed in $TNS_ADMIN or $GRID_HOME/network/admin directory to add encryption, i.e.
SQLNET.ENCRYPTION_TYPES_SERVER={3DES168}
SQLNET.ENCRYPTION_SERVER=REQUIRED
SOLUTION
Restore sqlnet.ora from backup and start ASM from sqlplus.