LB集群的架构和原理很简单,就是当用户的请求过来时,会直接分发到DirectorServer上,然后它把用户的请求根据设置好的调度算法,智能均衡地分发到后端真正服务器(realserver)上。为了避免不同机器上用户请求得到的数据不一样,需要用到了共享存储,这样保证所有用户请求的数据是一样的。
LVS是 Linux Virtual Server 的简称,也就是Linux虚拟服务器。这是一个由章文嵩博士发起的一个开源项目,它的官方网站是 http://www.linuxvirtualserver.org 现在 LVS 已经是 Linux 内核标准的一部分。使用 LVS可以达到的技术目标是:通过 LVS 达到的负载均衡技术和 Linux 操作系统实现一个高性能高可用的 Linux服务器集群,它具有良好的可靠性、可扩展性和可操作性。从而以低廉的成本实现最优的性能。LVS是一个实现负载均衡集群的开源软件项目,LVS架构从逻辑上可分为调度层、Server集群层和共享存储。
摘自 <http://www.cnblogs.com/liwei0526vip/p/6370103.html>
os | rhel7.2 |
lvs01 | 192.168.1.153 | keepalived \ ipvsadm |
lvs02 | 192.168.1.154 | keepalived \ ipvsadm |
vip | 192.168.1.160 |
|
realserver01 | 192.168.1.151 | nginx |
realserver02 | 192.168.1.152 | nginx |
1、初始化系统
# vi /etc/selinux/config
SELINUX=disabled |
# systemctl disable firewalld
# hostnamectl set-hostname xxx
# reboot
2、realserver Nginx 安装
# yum -y install gcc zlibzlib-devel openssl openssl-devel pcre pcre-devel wget
# cd /usr/local/src/
# wget http://nginx.org/download/nginx-1.11.12.tar.gz
# tar -zxf nginx-1.11.12.tar.gz
# cd nginx-1.11.12/
# ./configure --prefix=/usr/local/nginx
# make && make install
# echo new1 > /usr/local/nginx/html/index.html #realserver2 用 new2
# /usr/local/nginx/sbin/nginx # 启动
# curl 192.168.1.151
new1 |
# curl 192.168.1.152
new2 |
3、lvs 主机 安装 keepalived + ipvsadm
# yum -y install ipvsadm keepalived ipset
# vi /etc/keepalived/keepalived.conf # master 配置文件
! Configuration File for keepalived
global_defs { #notification_email { # acassen@firewall.loc # failover@firewall.loc # sysadmin@firewall.loc #} #notification_email_from Alexandre.Cassen@firewall.loc #smtp_server 127.0.0.1 #smtp_connect_timeout 30 router_id LVS_DEVEL #vrrp_skip_check_adv_addr #vrrp_strict #vrrp_garp_interval 0 #vrrp_gna_interval 0 }
vrrp_instance VI_1 { state MASTER interface eno16780032 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.160 } }
virtual_server 192.168.1.160 80 { delay_loop 6 lb_algo rr lb_kind DR persistence_timeout 50 protocol TCP
real_server 192.168.1.151 80 { weight 1 TCP_CHECK { connect_timeout 3 # nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.1.152 80 { weight 1 TCP_CHECK { connect_timeout 3 # nb_get_retry 3 delay_before_retry 3 connect_port 80 } } } |
# vi /etc/keepalived/keepalived.conf # backup 配置文件
! Configuration File for keepalived
global_defs { #notification_email { # acassen@firewall.loc # failover@firewall.loc # sysadmin@firewall.loc #} #notification_email_from Alexandre.Cassen@firewall.loc #smtp_server 127.0.0.1 #smtp_connect_timeout 30 router_id LVS_DEVEL #vrrp_skip_check_adv_addr #vrrp_strict #vrrp_garp_interval 0 #vrrp_gna_interval 0 }
vrrp_instance VI_1 { state BACKUP interface eno16780032 virtual_router_id 51 priority 50 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.160 } }
virtual_server 192.168.1.160 80 { delay_loop 6 lb_algo rr lb_kind DR persistence_timeout 50 protocol TCP
real_server 192.168.1.151 80 { weight 1 TCP_CHECK { connect_timeout 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.1.152 80 { weight 1 TCP_CHECK { connect_timeout 3 delay_before_retry 3 connect_port 80 } } } |
4、realserver 启动脚本
# vi /etc/rc.d/init.d/realserver.sh
#!/bin/bash #description: Config realserver
VIP=192.168.1.160
/etc/rc.d/init.d/functions
case "$1" in start) /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP /sbin/route add -host $VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "RealServer Start OK" ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac
exit 0 |
# chmod 777 /etc/rc.d/init.d/functions
# chmod +x /etc/rc.d/init.d/realserver.sh
# /etc/rc.d/init.d/realserver.sh start
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 192.168.1.160/32 brd 192.168.1.160 scope global lo:0 valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno16780032: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000 link/ether 00:50:56:a7:86:21 brd ff:ff:ff:ff:ff:ff inet 192.168.1.151/24 brd 192.168.1.255 scope global eno16780032 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fea7:8621/64 scope link valid_lft forever preferred_lft forever |
5、lvs 主机启动keepalived
#systemctl start keepalived && systemctl enable keepalived
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno16780032: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000 link/ether 00:50:56:a7:26:00 brd ff:ff:ff:ff:ff:ff inet 192.168.1.153/24 brd 192.168.1.255 scope global eno16780032 valid_lft forever preferred_lft forever inet 192.168.1.160/32 scope global eno16780032 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fea7:2600/64 scope link valid_lft forever preferred_lft forever |
# tailf /var/log/messages
Nov 6 21:33:32 mysql-01 systemd: Starting LVS and VRRP High Availability Monitor... Nov 6 21:33:32 mysql-01 Keepalived[2341]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5 -6-g6fa32f2 Nov 6 21:33:32 mysql-01 Keepalived[2341]: Unable to resolve default script username 'keepalived_scri pt' - ignoring Nov 6 21:33:32 mysql-01 Keepalived[2341]: Opening file '/etc/keepalived/keepalived.conf'. Nov 6 21:33:32 mysql-01 Keepalived[2342]: Starting Healthcheck child process, pid=2343 Nov 6 21:33:32 mysql-01 Keepalived[2342]: Starting VRRP child process, pid=2344 Nov 6 21:33:32 mysql-01 systemd: Started LVS and VRRP High Availability Monitor. Nov 6 21:33:32 mysql-01 Keepalived_healthcheckers[2343]: Opening file '/etc/keepalived/keepalived.co nf'. Nov 6 21:33:32 mysql-01 Keepalived_healthcheckers[2343]: Activating healthchecker for service [192.1 68.1.160]:80 Nov 6 21:33:32 mysql-01 Keepalived_healthcheckers[2343]: Activating healthchecker for service [192.1 68.1.160]:80 Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: Registering Kernel netlink reflector Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: Registering Kernel netlink command channel Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: Registering gratuitous ARP shared channel Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: Opening file '/etc/keepalived/keepalived.conf'. Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: VRRP_Instance(VI_1) removing protocol VIPs. Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: Using LinkWatch kernel netlink reflector... Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), f d(10,11)] Nov 6 21:33:32 mysql-01 Keepalived_vrrp[2344]: VRRP_Instance(VI_1) Transition to MASTER STATE Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: VRRP_Instance(VI_1) Entering MASTER STATE Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: VRRP_Instance(VI_1) setting protocol VIPs. Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eno16780032 for 192.168.1.160 Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:33 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:38 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:38 mysql-01 Keepalived_vrrp[2344]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eno16780032 for 192.168.1.160 Nov 6 21:33:38 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:38 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:38 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 Nov 6 21:33:38 mysql-01 Keepalived_vrrp[2344]: Sending gratuitous ARP on eno16780032 for 192.168.1.1 60 |
# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.160:80 rr persistent 50 -> 192.168.1.151:80 Route 1 0 0 -> 192.168.1.152:80 Route 1 0 0 |
6、测试
# curl 192.168.1.160
new1 |
# curl 192.168.1.160
new2 |
# 手动 down 一个 lvs 节点,查看 vip 是否切换,端口是否可用
# 手动 down 一个 nginx 节点,查看 ipvsadm -ln 是否自动删除该节点信息等等
# 过程略