CRS-2409:The clock on host rac2 is not synchronous with the mean cluster time. No action has been ta

环境介绍
aix 6.1
11.2.0.4 rac

现象：
集群alert日志中输出

CRS-2409:The clock on host rac2 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is ru
nning in observer mode.

解释
当ctssd运行在observe的模式中并且集群中节点之间通过ntp同步时间，当节点之间时差超过1ms的时候，ctssd就会报出这个这个消息，如果
集群中的时间是同步的，那么可以忽略这个告警。

问题排查
1、检查ctssd的状态
crsctl stat resource ora.ctssd -init

2、检查ntp的状态

# lssrc -ls xntpd
 Program name:    /usr/sbin/xntpd 
 Version:         3 
 Leap indicator:  00 (No leap second today.) 
 Sys peer:        no peer, system is insane 
 Sys stratum:     16 
 Sys precision:   -17 
 Debug/Tracing:   DISABLED 
 Root distance:   16.002151 
 Root dispersion: 0.006104 
 Reference ID:    192.168.11.19 
 Reference time:  e6af1dec.29a49000  Tue, Aug 23 2022 17:35:40.162 
 Broadcast delay: 0.003906 (sec) 
 Auth delay:      0.000122 (sec) 
 System flags:    pll monitor filegen  
 System uptime:   40004453 (sec) 
 Clock stability: 40.091385 (sec) 
 Clock frequency: 0.000000 (sec) 
 Peer: 192.168.11.19 
      flags: (configured)(preferred) 
      stratum:  1, version: 3 
      our mode: client, his mode: server 
Subsystem         Group            PID          Status 
 xntpd            tcpip            6488348      active
# 

3、检查集群的时间同步情况，确保ntp以-x启动

[grid@rac1:/home/grid]$ cluvfy comp clocksync -n all -verbose

Verifying Clock Synchronization across the cluster nodes 

Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed

Checking if CTSS Resource is running on all nodes...
Check: CTSS Resource running on all nodes
  Node Name                             Status                  
  ------------------------------------  ------------------------
  rac2                                passed                  
  rac1                                passed                  
Result: CTSS resource check passed


Querying CTSS for time offset on all nodes...
Result: Query of CTSS for time offset passed

Check CTSS state started...
Check: CTSS state
  Node Name                             State                   
  ------------------------------------  ------------------------
  rac2                                Observer                
  rac1                                Observer                
CTSS is in Observer state. Switching over to clock synchronization checks using NTP


Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP Configuration file check passed

Checking daemon liveness...

Check: Liveness for "xntpd"
  Node Name                             Running?                
  ------------------------------------  ------------------------
  rac2                                yes                     
  rac1                                yes                     
Result: Liveness check passed for "xntpd"
Check for NTP daemon or service alive passed on all nodes

Checking NTP daemon command line for slewing option "-x"
Check: NTP daemon command line
  Node Name                             Slewing Option Set?     
  ------------------------------------  ------------------------
  rac2                                yes                     
  rac1                                yes                     
Result: 
NTP daemon slewing option check passed

Checking NTP daemon's boot time configuration, in file "/etc/rc.tcpip", for slewing option "-x"

Check: NTP daemon's boot time configuration
  Node Name                             Slewing Option Set?     
  ------------------------------------  ------------------------
  rac2                                yes                     
  rac1                                yes                     
Result: 
NTP daemon's boot time configuration check for slewing option passed

Checking whether NTP daemon or service is using UDP port 123 on all nodes

Check for NTP daemon or service using UDP port 123
  Node Name                             Port Open?              
  ------------------------------------  ------------------------
  rac2                                yes                     
  rac1                                yes                     

Result: Clock synchronization check using Network Time Protocol(NTP) passed


Oracle Cluster Time Synchronization Services check passed

Verification of Clock Synchronization across the cluster nodes was successful. 
[grid@rac1:/home/grid]$ 

4、确认节点之间的时间是否有差距

[grid@rac2:/u01/app/11.2.0/grid/log/rac2]$ date;ssh rac1 date
Tue Nov 21 10:25:32 CST 2023
Tue Nov 21 10:25:32 CST 2023
[grid@rac2:/u01/app/11.2.0/grid/log/rac2]$ 

当xntpd不是运行在-x模式下的时候：
1、crsctl stop resource ora.ctssd -init
2、stopsrc -s xntpd
3、vi /etc/rc.tcpip 修改
start /usr/sbin/xntpd “$src_running” -a “-x”

4、启动ntpd
startsrc -s xntpd -a “-x”

5、启动ctssd
crsctl start resource ora.ctssd -init

6、校验ntp
cluvfy comp clocksync -n all -verbose