keepalived+squid高可用代理

1、机器规划

1.1 机器分配

文档是测试，使用了两台机器，实际可以是两台或者三台机器

IP	功能	配置	说明
10.11.123.11	squid	4C8G
10.11.123.12	squid	4C8G
10.11.123.13	Keepalived	2C4G
10.11.123.14	Keepalived	2C4G
10.11.123.15	Vip

1.2 前置操作

所有机器都操作

1.2.1 关闭防火墙

systemctl stop firewalld ; systemctl disable firewalld
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

1.2.2 关闭 swap 分区

swapoff -a && sysctl -w vm.swappiness=0
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab

1.2.3 安装命令

yum install net-tools -y

2、keepalived安装

2.1 master节点安装

10.11.123.13机器

2.1.1 软件安装

yum install -y keepalived

2.1.2 配置修改

! Configuration File for keepalived
global_defs {
   notification_email {
     testemail@qq.com
   }
   notification_email_from testemail@qq.com
   smtp_server 172.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface ens192
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.11.123.15
    }
}

virtual_server 10.11.123.15 3128 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 10.11.123.11 3128 {
        weight 1
        #SSL_GET {
        #    url {
        #      path /
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }

    real_server 10.11.123.12 3128 {
        weight 1 
        #SSL_GET {
        #    url {
        #      path / 
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }
}

2.1.3 启动服务

systemctl start keepalived
systemctl enable keepalived
systemctl restart keepalived
systemctl status keepalived

2.2 backup节点安装

10.11.123.14机器

2.2.1 软件安装

yum install -y curl gcc openssl-devel libnl3-devel net-snmp-devel
yum install -y keepalived

2.2.2 配置修改

! Configuration File for keepalived
global_defs {
   notification_email {
     testemail@qq.com
   }
   notification_email_from testemail@qq.com
   smtp_server 172.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens192
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.11.123.15
    }
}

virtual_server 10.11.123.15 3128 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 10.11.123.11 3128 {
        weight 1
        #SSL_GET {
        #    url {
        #      path /
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }

    real_server 10.11.123.12 3128 {
        weight 1 
        #SSL_GET {
        #    url {
        #      path / 
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }
}

2.2.3 启动服务

systemctl start keepalived
systemctl enable keepalived
systemctl restart keepalived
systemctl status keepalived

3、代理服务安装

3.1 安装squid

两个节点安装方式一样,这里使用Squid,使用yum方式安装Squid

第一步：安装
yum install -y squid

第二步：修改代理端口
vim /etc/squid/squid.conf
acl localnet src 192.168.0.0/16 #配置运行访问代理服务器的源
http_port 3128 #设置代理服务器监听端口，默认3128，建议修改，如果启用了防火墙需要开放监听端口的安全策略。

第三步：查看服务端口
netstat -tnpl |grep 3128

第四步：启动代理服务
systemctl start squid && systemctl enable squid 

3.2 配置vip

10.11.123.11和10.11.123.12都进行配置

vim /etc/init.d/realserver
chmod 755 /etc/init.d/realserver
service realserver start

realserver文件

SNS_VIP=10.11.123.15
/etc/rc.d/init.d/functions
case "$1" in
start)
       ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
       /sbin/route add -host $SNS_VIP dev lo:0
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       sysctl -p >/dev/null 2>&1
       echo "RealServer Start OK"
       ;;
stop)
       ifconfig lo:0 down
       route del $SNS_VIP >/dev/null 2>&1
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       echo "RealServer Stoped"
       ;;
*)
       echo "Usage: $0 {start|stop}"
       exit 1
esac
exit 0

4、应用容器配置

4.1 配置环境变量

创建工作负载的时候设置环境变量，下面使用nginx为例

ip为keepalived的ip，端口为squid的端口

4.2 验证代理

查看pod的环境变量

环境变量中显示：

http_proxy=http://10.11.123.15:3128

https_proxy=https://10.11.123.15:3128

测试访问外网是否走代理

状态码为200，并且走代理，返回界面

4.3 验证代理高可用

1、curl baidu.com看返回结果

2、停止一台机器的squid服务，在curl看下结果

3、结论

当2台中的一台squid停止服务，keepalived会进行健康检查，发现服务不通回调用另一个机器的服务，实现高可用。