redhat 7.3 ppc64le环境安装Keepalived及遇到问题

因为配置了本地YUMi源，最开始我是直接YUM INSTALL KEEPALIVED安装的

但装出来的版本是1.2.X

修改/etc/keepalived/keepalived配置文件，启服务正常。

但是不能用，在/var/log/message 日志中查看

May 29 11:49:45 PHY153 Keepalived[12064]: Starting Keepalived v1.2.13 (07/01,2016)
May 29 11:49:45 PHY153 Keepalived[12065]: Starting Healthcheck child process, pid=12066
May 29 11:49:45 PHY153 systemd: Started LVS and VRRP High Availability Monitor.
May 29 11:49:45 PHY153 Keepalived[12065]: Starting VRRP child process, pid=12067
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Netlink: error: message truncated
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Netlink: error: message truncated
May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Netlink: error: message truncated
May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Netlink: error: message truncated
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Registering Kernel netlink reflector
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Registering Kernel netlink command channel
May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Registering Kernel netlink reflector
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Registering gratuitous ARP shared channel
May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Registering Kernel netlink command channel
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Opening file '/etc/keepalived/keepalived.conf'.
May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Opening file '/etc/keepalived/keepalived.conf'.
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Cant find interface bond.pub for vrrp_instance VI_1 !!!
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Configuration error: VRRP definition must belong to an interface
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: Default interface eth0 does not exist and no interface specified. Skip VRRP address.
May 29 11:49:45 PHY153 Keepalived_vrrp[12067]: VRRP_Instance(VI_1) Unknown interface !
May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Configuration is using : 6718 Bytes

May 29 11:49:45 PHY153 Keepalived_healthcheckers[12066]: Using LinkWatch kernel netlink reflector...

以上是报错信息，无耐呀，第一次安装只能百度关键字Netlink: error: message truncated

中文网站基本没有这个报错信息的

再搜索keepalived Netlink: error: message truncated

找到官网了：http://www.keepalived.org/changelog.html

进去搜索关键字：Netlink: error: message truncated

只有一段话：

Resolve "Netlink: error: message truncated" messages.
  On systems with a page size larger than 4096 keepalived may report:
  "Netlink: error: message truncated" messages
  This error was reported on a ppc64le in an OpenStack/Nutron environment.
  Ppc64le is using a 64k pages size. I found that keepalived's netlink recvmsg
  buffer was too small causing messages to be truncated. The size of the read
  buffer for the netlink socket should be based on page size however, it should
  not exceed 8192. See the comment in the patch.

大概意思是页面文件大小不一样，而且只会发生在ppc64le环境下，这个问题只能在后续补丁中解决。

这个版本是肯定不行了，通过编译方式下载了更高一点的1.3.X，一样报这个错误，索性直接下载2.0.0

以后主要是以编译方式安装Keepalived2.0的内容：全部在ROOT用户下执行

在官网自己去下载2.0版本，放到服务器，解压好，我是直接解压到/home/用户/keepalived2.0目录的

进到目录里边执行【/usr/local/keepalived 这个文件夹是自动建的还是我手动建的忘了】

./configure --prefix=/usr/local/keepalived

最开始提示YES，最后报错：

configure: error: 
  !!! OpenSSL is not properly installed on your system. !!!

  !!! Can not include OpenSSL headers files.            !!!

OpenSSL包有问题？

网上提示：yum -y install openssl-devel

可以解决问题，我rpm -qa|grep openssl-devel  确实没有这个包，但我yum -y install openssl-devel 提示一些问题，装不上。

因为我有ppc64le下的所有RPM包 我YUM 安装的是1.0.2版的，而我自己存的rpm包是1.0.1版本的

所以把现在服务器openssl包都卸载的rpm -e *** --nodeps

基本都要强制卸载，之后再rpm -qa|grepp openssl

没有了之后再手动安装，安装有顺序

[root@PHY154 ppcrpm]# rpm -ivh openssl-1.0.1e-42.el7.9.ppc64le.rpm  --把OPENSSL装上，提示依赖包
warning: openssl-1.0.1e-42.el7.9.ppc64le.rpm: Header V4 RSA/SHA1 Signature, key ID f533f4fa: NOKEY
error: Failed dependencies:
        libcrypto.so.10()(64bit) is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
        libcrypto.so.10(OPENSSL_1.0.1)(64bit) is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
        libcrypto.so.10(OPENSSL_1.0.1_EC)(64bit) is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
        libcrypto.so.10(libcrypto.so.10)(64bit) is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
        libssl.so.10()(64bit) is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
        libssl.so.10(libssl.so.10)(64bit) is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
        openssl-libs(ppc-64) = 1:1.0.1e-42.el7.9 is needed by openssl-1:1.0.1e-42.el7.9.ppc64le
[root@PHY154 ppcrpm]# rpm -ivh openssl-libs-1.0.1e-42.el7.9.ppc64le.rpm  --先装这个，
warning: openssl-libs-1.0.1e-42.el7.9.ppc64le.rpm: Header V4 RSA/SHA1 Signature, key ID f533f4fa: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:openssl-libs-1:1.0.1e-42.el7.9   ################################# [100%]
[root@PHY154 ppcrpm]# rpm -ivh openssl-devel-1.0.1e-42.el7.9.ppc64le.rpm         
warning: openssl-devel-1.0.1e-42.el7.9.ppc64le.rpm: Header V4 RSA/SHA1 Signature, key ID f533f4fa: NOKEY
error: Failed dependencies:
        krb5-devel(ppc-64) is needed by openssl-devel-1:1.0.1e-42.el7.9.ppc64le
[root@PHY154 ppcrpm]# rpm -ivh openssl-1.0.1e-42.el7.9.ppc64le.rpm  --再装
warning: openssl-1.0.1e-42.el7.9.ppc64le.rpm: Header V4 RSA/SHA1 Signature, key ID f533f4fa: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:openssl-1:1.0.1e-42.el7.9        ################################# [100%]
[root@PHY154 ppcrpm]# 

安装好了，再rpm -qa|grep openssl

一共显示3个：

[root@XXXX]# rpm -qa|grep openssl
openssl-libs-1.0.1e-60.el7.ppc64le
openssl-devel-1.0.1e-60.el7.ppc64le

openssl-1.0.1e-60.el7.ppc64le

再进入到Keepalived目录

执行：./configure --prefix=/usr/local/keepalived    

一路YES  就OK了。

执行：make

执行：make install

基本算是安装成功了。

记得在/etc/keepalived/ 目录建立 keepalived.conf 文件，因为KEEPALIVED会调用这个配置文件，在里边配置好参数。

我的主服务器配置如下：

! Configuration File for keepalived
global_defs {
   router_id LVS_DEVEL
}


vrrp_script chk_http_port {
    script "/opt/nginx_pid.sh"
    interval 2
    weight 2
}


vrrp_instance VI_1 {
    state MASTER
    interface bond.pub
    virtual_router_id 53
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    track_script {
        chk_http_port
    }
    virtual_ipaddress {
        10.10.10.10
    }

}

我的备服务器配置如下：

! Configuration File for keepalived
global_defs {
   router_id LVS_DEVEL
}


vrrp_script chk_http_port {
    script "/opt/nginx_pid.sh"
    interval 2
    weight 2
}


vrrp_instance VI_1 {
    state BACKUP
    interface bond.pub
    virtual_router_id 53
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    track_script {
        chk_http_port
    }
    virtual_ipaddress {
        10.10.10.10
    }

}

配置完成之后，重启keepalived服务。 

service keepalived restart

之后 ping 10.10.10.10 

arp 命令看一下，MAC地址是主机的，

在主机pkill keepalived 将进程杀死，此时VIP就自动飘到备机

之后 ping 10.10.10.10 

arp 命令看一下，MAC地址是备机的

此时算是正常了，我自己PC机一直 PING VIP，查看网络情况，发现丢包率非常大，20%

我以为是自己网络不好，但打开网页确非常流畅。

PING BAIDU.COM 确一切正常，再PING VIP一样丢包。

VIP肯定有问题，我单独PING 主机，备机。特别正常。

我将主机KEEPALIVED进程杀掉，让VIP飘到备机，再PING VIP正常。

将主机KEEPALLIVED服务起来，VIP自动飘到主机，丢包开始。

看日志/var/log/message里边有这样段，只要出现这段，肯定丢包。

VI_1) ip address associated with VRID 53 not present in MASTER advert

百度之后找到两段话：在同一网段内非同一套keepalive集群中的virtual_router_id 值不能相同，如果相同会在messages中收到VRRP错误包 ，所以需要更改 virual_router_id。

因为同一集群的keepalived的主、备机的virtual_router_id 必须相同,取值0-255 但是同一内网中不应有相同virtual_router_id的集群。

最开始看到这段话就懵了，我的两台服务器都在同一网段，所以virtual_router_id 值不能相同？我的两台服务器是同一集群，所以virtual_router_id 必须相同,取值0-255 ？

#我将备机的KEEPALIVED进程杀掉后，观看message日志，发现报错依旧，才意识到哪里不对，原来是局域网里还有其它KEEPALIVED也用在这个virtual_router_id....

#最后总结，意思就是你所处的局域网内，有其它机器也在用KEEPALIVED，而且它的virtual_router_id和你配置的一样，只需要将自己的主备的virtual_router_id改成其它的就好。它用53，我就用其它的，我把两台服务器virtual_router_id改成64，重启两台服务器服务。

最后查看日志，测试网络正常。主备机切换也正常。至此，问题全部解决。