#一、Haproxy 基础
软件:haproxy
—主要是做负载均衡的7层,也可以做4层负载均衡
apache也可以做7层负载均衡,但是很麻烦。实际工作中没有人用。
负载均衡是通过OSI协议对应的
7层负载均衡:用的7层http协议,
4层负载均衡:用的是tcp协议加端口号做的负载均衡
ha-proxy概述:
ha-proxy是一款高性能的负载均衡软件。因为其专注于负载均衡这一些事情,因此与nginx比起来在负载均衡这件事情上做更好,更专业。
ha-proxy的特点:
ha-proxy 作为目前流行的负载均衡软件,必须有其出色的一面。下面介绍一下ha-proxy相对LVS,Nginx等负载均衡软件的优点。
- 支持 tcp / http 两种协议层的负载均衡,使得其负载均衡功能非常丰富。
- 支持8种左右的负载均衡算法,尤其是在http模式时,有许多非常实在的负载均衡算法,适用各种需求。
- 性能非常优秀,基于单进程处理模式(和Nginx类似)让其性能卓越。
- 拥有一个功能出色的监控页面,实时了解系统的当前状况。
- 功能强大的ACL支持,给用户极大的方便。
haproxy算法:
roundrobin
:
基于权重进行轮询,在服务器的处理时间保持均匀分布时,这是最平衡,最公平的算法.此算法是动态的,这表示其权重可以在运行时进行调整.static-rr
:
基于权重进行轮询,与roundrobin类似,但是为静态方法,在运行时调整其服务器权重不会生效.不过,其在后端服务器连接数上没有限制leastconn
:
新的连接请求被派发至具有最少连接数目的后端服务器.
#二、Haproxy实现七层负载
###1. 准备工作
- 准备四台虚拟机
- 关闭防火墙以及selinux
两台RS服务器安装nginx:配置显示不同内容方便测试
192.168.181.129
192.168.181.130两台Haproxy(主/备):
192.168.181.128
192.168.181.145
- 可以给集群中所有主机做解析:
[root@master ~]# cat /etc/hosts 127.0.0.1 localhost 192.168.181.128 master 192.168.181.145 slave 192.168.181.129 nginx1 192.168.181.130 nginx2
###2. 主/备调度器安装配置 Haproxy
[root@master ~]# yum -y install haproxy
[root@slave ~]# yum -y install haproxy
[root@master ~]# cp -rf /etc/haproxy/haproxy.cfg{,.bak}
[root@master ~]# sed -i -r '/^[ ]*#/d;/^$/d' /etc/haproxy/haproxy.cfg
[root@master ~]# vim /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local2 info #日志服务器
pidfile /var/run/haproxy.pid #pid文件
maxconn 4000 #最大连接数
user haproxy #用户
group haproxy #组
daemon #以后台形式运行ha-proxy
nbproc 1 #工作进程数量 cpu内核是几就写几
defaults #defaults字段用于为其他配置段提供默认参数
mode http #工作模式 http ,tcp 是 4 层,http是 7 层
log global
retries 3 #健康检查。3次连接失败就认为服务器不可用,主要通过后面的check检查
option redispatch #服务不可用后重定向到其他健康服务器。
maxconn 4000 #优先级中
contimeout 5000 #ha服务器与后端服务器连接超时时间,单位毫秒ms
clitimeout 50000 #客户端超时
srvtimeout 50000 #后端服务器超时
listen stats #listen是frontend和backend的结合体
bind *:81
stats enable
stats uri /haproxy #使用浏览器访问192.168.181.128/haproxy,可以看到服务器状态
stats auth chen:123 #用户认证,客户端使用elinks浏览器的时候不生效
frontend web #frontend是虚拟服务VIrtual Server
mode http
bind *:80 #监听哪个ip和什么端口
option httplog #日志类别 http 日志格式
acl html url_reg -i \.html$ #1.访问控制列表名称html。规则要求访问以html结尾的url(可选)
use_backend httpservers if html #2.如果满足acl html规则,则推送给后端服务器httpservers
default_backend httpservers #默认使用的服务器组
backend httpservers #bacaend是真实服务器Real Server #后面名字要与上面的名字必须一样
balance roundrobin #负载均衡的方式
server http1 192.168.181.129:80 maxconn 2000 weight 1 check inter 1s rise 2 fall 2
server http2 192.168.181.130:80 maxconn 2000 weight 1 check inter 1s rise 2 fall 2
#slave服务器如上配置
[root@slave ~]# rm -rf /etc/haproxy/*
[root@master ~]# scp /etc/haproxy/haproxy.cfg 192.168.181.144:/etc/haproxy/
[root@master ~]# systemctl start haproxy
[root@master ~]# systemctl enable haproxy
[root@slave ~]# systemctl start haproxy
[root@slave ~]# systemctl enable haproxy
###3. 访问测试
主:
备:
页面主要参数解释:
Queue
Cur: current queued requests #当前的队列请求数量
Max:max queued requests #最大的队列请求数量
Limit: #队列限制数量
Errors
Req:request errors #错误请求
Conn:connection errors #错误的连接
Server列表:
Status:状态,包括up(后端机活动)和down(后端机挂掉)两种状态
LastChk: 持续检查后端服务器的时间
Wght: (weight) : 权重
如果出现bind失败的报错,执行下列命令
setsebool -P haproxy_connect_any=1
#三、Keepalived实现调度器HA
注:主/备调度器均能够实现正常调度
vip:192.168.181.88
###1. 主/备调度器安装配置Keepalived
[root@master ~]# yum -y install keepalived
[root@slave ~]# yum -y install keepalived
[root@master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id director1
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 80
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.181.88/24
}
}
[root@slave ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id directory2
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
nopreempt
virtual_router_id 80
priority 50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.181.88/24
}
}
[root@master ~]# systemctl start keepalived
[root@master ~]# systemctl enable keepalived
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:01:30:e0 brd ff:ff:ff:ff:ff:ff
inet 192.168.181.128/24 brd 192.168.181.255 scope global dynamic ens33
valid_lft 1303sec preferred_lft 1303sec
inet 192.168.181.88/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe01:30e0/64 scope link
valid_lft forever preferred_lft forever
[root@slave ~]# systemctl start keepalived
[root@slave ~]# systemctl enable keepalived
###2. 访问测试
此时可以正常访问
将keepalived主节点关机测试vip是否漂移
[root@master ~]# init 0
[root@slave ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:48:1a:fe brd ff:ff:ff:ff:ff:ff
inet 192.168.181.145/24 brd 192.168.181.255 scope global dynamic ens33
valid_lft 1495sec preferred_lft 1495sec
inet 192.168.181.88/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::9db3:1c86:27b7:1225/64 scope link
valid_lft forever preferred_lft forever
###3. 对调度器Haproxy健康检查(可选)
两台机器都做,让Keepalived以一定时间间隔执行一个外部脚本,脚本的功能是当Haproxy失败,则关闭本机的Keepalived
[root@master ~]# cat /etc/keepalived/check_haproxy_status.sh
#!/bin/bash
/usr/bin/curl -I http://localhost &>/dev/null
if [ $? -ne 0 ];then
# /etc/init.d/keepalived stop
systemctl stop keepalived
fi
[root@master ~]# chmod a+x /etc/keepalived/check_haproxy_status.sh
[root@master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id director1
}
vrrp_script check_haproxy {
script "/etc/keepalived/check_haproxy_status.sh"
interval 5
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 80
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.181.88/24
}
track_script {
check_haproxy
}
[root@slave ~]# cat /etc/keepalived/check_haproxy_status.sh
#!/bin/bash
/usr/bin/curl -I http://localhost &>/dev/null
if [ $? -ne 0 ];then
# /etc/init.d/keepalived stop
systemctl stop keepalived
fi
[root@slave ~]# chmod a+x /etc/keepalived/check_haproxy_status.sh
[root@slave ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id directory2
}
vrrp_script check_haproxy {
script "/etc/keepalived/check_haproxy_status.sh"
interval 5
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
nopreempt
virtual_router_id 80
priority 50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.181.88/24
}
track_script {
check_haproxy
}
}
[root@master keepalived]# systemctl restart keepalived
[root@slave keepalived]# systemctl restart keepalived
注:必须先启动haproxy,再启动keepalived
###4. 配置haproxy的日志:需要打开注释并添加
[root@master ~]# vim /etc/rsyslog.conf
# Provides UDP syslog reception #由于haproxy的日志是用udp传输的,所以要启用rsyslog的udp监听
$ModLoad imudp
$UDPServerRun 514
找到 #### RULES #### 下面添加
local2.* /var/log/haproxy.log
[root@master ~]# systemctl restart rsyslog
[root@master ~]# systemctl restart haproxy
[root@master ~]# tail -f /var/log/haproxy.log
Mar 18 21:25:46 localhost haproxy[28531]: Proxy stats started.
Mar 18 21:25:46 localhost haproxy[28531]: Proxy web started.
Mar 18 21:25:46 localhost haproxy[28531]: Proxy httpservers started.
Mar 18 21:26:13 localhost haproxy[28532]: 192.168.181.1:54161 [18/Mar/2020:21:26:13.355] web httpservers/http1 7/0/2/1/10 200 845 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.1"
Mar 18 21:26:13 localhost haproxy[28532]: 192.168.181.1:54161 [18/Mar/2020:21:26:13.366] web httpservers/http2 601/0/0/3/604 404 3823 - - ---- 2/2/0/0/0 0/0 "GET /favicon.ico HTTP/1.1"
Mar 18 21:26:16 localhost haproxy[28532]: 192.168.181.1:54161 [18/Mar/2020:21:26:13.971] web httpservers/http1 2830/0/1/0/2831 304 175 - - ---- 2/2/0/1/0 0/0 "GET / HTTP/1.1"
#四、Haproxy 实现四层负载
两台haproxy配置文件:
[root@ha-proxy-master ~]# cat /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local2
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
nbproc 1
defaults
mode http
log global
option redispatch
retries 3
maxconn 4000
contimeout 5000
clitimeout 50000
srvtimeout 50000
listen stats
bind *:81
stats enable
stats uri /haproxy
stats auth chen:123
frontend web
mode http
bind *:80
option httplog
default_backend httpservers
backend httpservers
balance roundrobin
server http1 192.168.181.129:80 maxconn 2000 weight 1 check inter 1s rise 2 fall 2
server http2 192.168.181.130:80 maxconn 2000 weight 1 check inter 1s rise 2 fall 2
listen mysql
bind *:3306
mode tcp
balance roundrobin
server mysql1 192.168.181.129:3306 weight 1 check inter 1s rise 2 fall 2
server mysql2 192.168.181.130:3306 weight 1 check inter 1s rise 2 fall 2
inter表示健康检查的间隔,单位为毫秒 可以用1s等,fall代表健康检查失败2回后放弃检查。rise代表连续健康检查成功2此后将认为服务器可用。默认的,haproxy认为服务时永远可用的,除非加上check让haproxy确认服务是否真的可用。
找一台机器做为客户端去测试,在测试的时候注意mysql的远程登录权限