1、介绍:
【HAProxy】是高性能的代理服务器,其可以提供7层和4层代理,具有healthcheck,负载均衡等多种特性,性能卓越
【KeepAlived】是一个高可用方案,通过VIP(即虚拟IP)和心跳检测来实现高可用,也通过该vip实现访问。其原理是存在一组(两台)服务器
默认情况下Master会绑定VIP到自己的网卡上,对外提供服务。如果Backup发现Master宕机,那么Backup会发送ARP包到网关,把VIP绑定到自己的网卡,此时Backup对外提供服务,实现自动化的故障转移,当Master恢复的时候会重新接管服务。
2、环境:
web1:192.168.1.78
web2:192.168.1.241
web3:192.168.1.133
web4:192.168.1.244
haproxy+keepalived1:192.168.1.22
haproxy+keepalived1:192.168.1.9
vip1:192.168.1.189(www.inbank.com)
vip2:192.168.1.199(p_w_picpath.inbank.com)
3、需求:
默认情况下,第一台负载均衡器主要分发www.baison.com.cn的请求,第二台负载均衡器主要分发img.baison.com.cn的请求。任意一台宕机都不会影响网站分发。这样不会导致服务器浪费。
4、keepavlied+haproxy安装略,直接进入haproxy和keepalived配置
【192.168.1.22】上的haproxy配置:
[root@master etc]# cat /usr/local/haproxy/etc/haproxy.cfg |grep -v "#"|sed '/^$/d'
global
log 127.0.0.1 local1 notice
maxconn 4096
chroot /usr/share/haproxy
uid 99
gid 99
daemon
pidfile /usr/local/haproxy/haproxy.pid
defaults
log global
mode http
retries 3
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
stats uri /haproxy-stats
balance roundrobin
frontend WEB_SITE
bind :80
acl web hdr(host) -i www.inbank.com
acl img hdr(host) -i p_w_picpath.inbank.com
use_backend webserver if web
use_backend imgserver if img
backend webserver
mode http
balance roundrobin
server web_1 192.168.1.78:80 check inter 2000 fall 5 weight 1
server web_2 192.168.1.241:80 check inter 2000 fall 5 weight 1
backend imgserver
mode http
option httpchk /index.php
balance roundrobin
server web_1 192.168.1.133:80 check inter 2000 fall 5 weight 1
server web_1 192.168.1.244:80 check inter 2000 fall 5 weight 1
192.168.1.22上的keepalived配置
[root@master keepalived]# cat keepalived.conf|grep -v "#"|sed '/^$/d'
!Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script chk_haproxy {
script "/usr/local/keepalived/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 88
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 5555
}
track_script {
chk_haproxy
}
virtual_ipaddress {
192.168.1.189
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 89 ---#id不能跟VI_1一样
priority 99---#
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.199
}
}
检测脚本,为了防止haproxy服务关闭导致keepalived不自动切换。
#vim /usr/local/keepalived/check_haproxy.sh
#!/bin/bash
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/etc/haproxy.cfg
fi
sleep 2
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
/etc/init.d/keepalived stop
fi
4、启动keepalived服务和haproxy,然后查看日志,看看是否有2个vip
tail -n 30 /var/log/messages
Oct 23 13:49:13 master Keepalived_vrrp: VRRP_Instance(VI_2) Entering BACKUP STATE
Oct 23 13:49:13 master Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(7,8)]
Oct 23 13:49:13 master Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Oct 23 13:49:14 master Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Oct 23 13:49:14 master Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Oct 23 13:49:14 master Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.189
Oct 23 13:49:14 master avahi-daemon[2879]: Registering new address record for 192.168.1.189 on eth0.
Oct 23 13:49:19 master Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.189
##从192.168.1.22上可以看出,VI_2已经进入了backup角色,VI_1已经进入了master,并且已经绑定了192.168.1.189在eth0网卡
[root@master keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:06:ed:78 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.22/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.189/32 scope global eth0
inet6 fe80::20c:29ff:fe06:ed78/64 scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
####从 192.168.1.9 haproxy和keepalived配置
haproxy不变,只是keepalived配置里的角色,权限需要跟master兑换下就可以了
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 88
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 5555
}
track_script {
chk_haproxy
}
virtual_ipaddress {
192.168.1.189
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 89
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.199
}
}
启动haproxy和keepalived服务,查看vip是否已经绑定在eth0网卡
Oct 23 14:00:26 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Oct 23 14:00:26 localhost Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Oct 23 14:00:27 localhost Keepalived_vrrp: VRRP_Instance(VI_2) Transition to MASTER STATE
Oct 23 14:00:28 localhost Keepalived_vrrp: VRRP_Instance(VI_2) Entering MASTER STATE
Oct 23 14:00:28 localhost Keepalived_vrrp: VRRP_Instance(VI_2) setting protocol VIPs.
Oct 23 14:00:28 localhost Keepalived_vrrp: VRRP_Instance(VI_2) Sending gratuitous ARPs on eth0 for 192.168.1.199
[root@localhost keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:2b:be:1a brd ff:ff:ff:ff:ff:ff
inet 192.168.1.9/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.199/32 scope global eth0
inet6 fe80::20c:29ff:fe2b:be1a/64 scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
四、web上的配置
在对应web上新增相关虚拟机 www.inbank.com,p_w_picpath.inbank.com
此处为了实验方便,就直接yum 安装了apache,在apache配置上的配置如下:
##web1 web2
NameVirtualHost *:80
<VirtualHost *:80>
ServerAdmin webmaster@dummy-host.example.com
DocumentRoot /var/www/html/inbank
ServerName www.inbank.com
ErrorLog logs/dummy-host.example.com-error_log
CustomLog logs/dummy-host.example.com-access_log common
</VirtualHost>
###web3 web4类似
在客户端修改hosts,新增以下2条记录
192.168.1.189 www.inbank.com
192.168.1.199 p_w_picpath.inbank.com
然后分别访问这2个网址,如下:
[root@master keepalived]# for i in `seq 1 4`;do curl http://p_w_picpath.inbank.com;done
img_133
img_244
img_133
img_244
[root@master keepalived]# for i in `seq 1 4`;do curl http://www.inbank.com;done
inbank_78
inbank_241
inbank_78
inbank_241
--如果出现以上的话,那就木有问题了
接下来测试高可用,停止master上的keepalived服务
[root@master keepalived]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]
然后查看backup上的状态
Oct 23 14:00:33 localhost Keepalived_vrrp: VRRP_Instance(VI_2) Sending gratuitous ARPs on eth0 for 192.168.1.199
Oct 23 14:08:04 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Oct 23 14:08:05 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Oct 23 14:08:05 localhost Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Oct 23 14:08:05 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.189
说明BACKUP已经接替MASTER了,然后再访问这2个网址,没问题的话,就大功告成了
##查看haproxy web 监控页面
http://192.168.1.189/haproxy-stats
或者http://192.168.1.199/haproxy-stats
#另,内核优化:
#haproxy+keepalived做前端,基本是tcp相关的内核优化,如下优化是一朋友线上环境用的,同样也适合lvs
net.ipv4.tcp_fin_timeout = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_keepalive_time = 600
net.ipv4.ip_local_port_range = 4000 65000
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.route.gc_timeout = 100
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_synack_retries = 1
net.ipv4.ip_conntrack_max = 25000000
net.ipv4.netfilter.ip_conntrack_max=25000000
net.ipv4.netfilter.ip_conntrack_max=6553600
#####【另】
1、在此HAProxy+Keepalivp负载均衡高可用架构中,我们是如何解决session的问题呢?我们这里采用的是它自身的balance source机制,它跟Nginx的ip_hash机制原理类似,是让客户机访问时始终访问后端的某一台真实的web服务器,这样让session就固定下来了;
2、option httpchk HEAD /index.jsp HTTP/1.0 是网页监控,如果HAProxy检测不到Web的根目录下没有index.jsp,就会产生503报错。
3、有网友配置HAProxy时喜欢用listen IP:80这样的格式,这样其实不好,做负载均衡高可用时由于从机分配不到VIP地址,会导致从机启动不了,我建议用bind *:80的方式代替。
转载于:https://blog.51cto.com/inbank2012/1317075