SUSE 15 HA中的节点重启后集群服务不正常,集群处于“partition WITHOUT quorum”的状态,同时两个节点都提示另一个节点的状态是“UNCLEAN (offline)”,如下所示:
节点1的crm status输出:
hanadb01:~ # crm status
Status of pacemakerd: 'Pacemaker is running' (last updated 2024-05-29 09:18:06 +08:00)
Cluster Summary:
* Stack: corosync
* Current DC: hanadb01 (version 2.1.5+20221208.a3f44794f-150500.4.9-2.1.5+20221208.a3f44794f) - partition WITHOUT quorum
* Last updated: Wed May 29 09:18:07 2024
* Last change: Wed May 29 09:11:22 2024 by hacluster via crmd on hanadb01
* 2 nodes configured
* 7 resource instances configured
Node List:
* Node hanadb02: UNCLEAN (offline)
* Online: [ hanadb01 ]
Full List of Resources:
* stonith-sbd (stonith:external/sbd): Stopped
* Clone Set: cln_SAPHanaTopology_HA1_HDB10 [rsc_SAPHanaTopology_HA1_HDB10]:
* Stopped: [ hanadb01 hanadb02 ]
* Clone Set: msl_SAPHana_HA1_HDB10 [rsc_SAPHana_HA1_HDB10] (promotable):
* Stopped: [ hanadb01 hanadb02 ]
* rsc_ip_HA1_HDB10 (ocf::heartbeat:IPaddr2): Stopped
* rsc_ip_HA1_HDB10_readenabled (ocf::heartbeat:IPaddr2): Stopped
节点2的crm status输出:
hanadb02:~ # crm status
Status of pacemakerd: 'Pacemaker is running' (last updated 2024-05-29 09:26:45 +08:00)
Cluster Summary:
* Stack: corosync
* Current DC: hanadb02 (version 2.1.5+20221208.a3f44794f-150500.4.9-2.1.5+20221208.a3f44794f) - partition WITHOUT quorum
* Last updated: Wed May 29 09:26:46 2024
* Last change: Wed May 29 09:21:57 2024 by root via cibadmin on hanadb02
* 2 nodes configured
* 7 resource instances configured
Node List:
* Node hanadb01: UNCLEAN (offline)
* Online: [ hanadb02 ]
Full List of Resources:
* stonith-sbd (stonith:external/sbd): Stopped
* Clone Set: cln_SAPHanaTopology_HA1_HDB10 [rsc_SAPHanaTopology_HA1_HDB10]:
* Stopped: [ hanadb01 hanadb02 ]
* Clone Set: msl_SAPHana_HA1_HDB10 [rsc_SAPHana_HA1_HDB10] (promotable):
* Stopped: [ hanadb01 hanadb02 ]
* rsc_ip_HA1_HDB10 (ocf::heartbeat:IPaddr2): Stopped
* rsc_ip_HA1_HDB10_readenabled (ocf::heartbeat:IPaddr2): Stopped
查看corosync服务的运行状态,提示“ [TOTEM ] Digest does not match”的错误:
hanadb02:~ # systemctl status corosync
● corosync.service - Corosync Cluster Engine
Loaded: loaded (/usr/lib/systemd/system/corosync.service; disabled; vendor preset: disabled)
Active: active (running) since Wed 2024-05-29 09:23:34 CST; 4min 27s ago
Docs: man:corosync
man:corosync.conf
man:corosync_overview
Process: 1954 ExecStart=/usr/share/corosync/corosync start (code=exited, status=0/SUCCESS)
Main PID: 2166 (corosync)
Tasks: 2
CGroup: /system.slice/corosync.service
└─ 2166 corosync
May 29 09:28:00 hanadb02 corosync[2166]: [TOTEM ] Invalid packet data
May 29 09:28:00 hanadb02 corosync[2166]: [TOTEM ] Digest does not match
May 29 09:28:00 hanadb02 corosync[2166]: [TOTEM ] Received message has invalid digest... ignoring.
May 29 09:28:00 hanadb02 corosync[2166]: [TOTEM ] Invalid packet data
May 29 09:28:01 hanadb02 corosync[2166]: [TOTEM ] Digest does not match
May 29 09:28:01 hanadb02 corosync[2166]: [TOTEM ] Received message has invalid digest... ignoring.
May 29 09:28:01 hanadb02 corosync[2166]: [TOTEM ] Invalid packet data
May 29 09:28:01 hanadb02 corosync[2166]: [TOTEM ] Digest does not match
May 29 09:28:01 hanadb02 corosync[2166]: [TOTEM ] Received message has invalid digest... ignoring.
May 29 09:28:01 hanadb02 corosync[2166]: [TOTEM ] Invalid packet data
经过检查,发现集群两台主机的/etc/corosync/authkey文件内容不一致,将其中一个节点的文件拷贝到另外一个节点,然后重启corosync服务,集群恢复正常。