haproxy对websocket的负载
用习惯了HaProxy,最近用到关于WebSocket的负载上也涉及到这块,做个记录。
这里以简单的 kube-apiserver 3master 转发http为例子,实际直接4层转发即可
单纯使用ha的tcp或者http负载,会出现链接断开的情况,此处需重点关注Connection: Upgrade头,会让client端知道server端会改变协议,变成如Upgrade: websocketheader中所述的协议。若服务端提供,则会返回:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: tD0l5WXr+s0lqKRayF9ABifcpzY=
Sec-WebSocket-Protocol: echo-protocol
下图为整体的交互图:
HaProxy对websocket的负载
具体配置
#**** WSS ****#
frontend WSS_SSL
bind *:443 ssl crt /etc/ssl/name.pem
mode http
## routing based on websocket protocol header
acl hdr_connection_upgrade hdr(Connection) -i upgrade
acl hdr_upgrade_websocket hdr(Upgrade) -i websocket
use_backend wss_srv if hdr_connection_upgrade hdr_upgrade_websocket
backend wss_srv
balance roundrobin
cookie SERVERID
server host1 192.168.1.10:80 cookie host1 maxconn 50000 check inter 10s rise 3 fall 3
下面为http 转发例子。
# HAProxy Configure /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
log global
option httplog
option dontlognull
timeout connect 5000
timeout client 5000
timeout server 5000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
listen admin_stats
bind *:1080 #设置Frontend和Backend的组合体,监控组的名称,按需要自定义名称
mode http #http的7层模式
maxconn 10 #默认的最大连接数
stats refresh 30s #统计页面自动刷新时间
stats uri /stats #统计页面url
stats auth admin:admin #设置监控页面的用户和密码:admin,可以设置多个用户名
#stats hide-version #隐藏统计页面上HAProxy的版本信息
stats admin if TRUE #设置手工启动/禁用,后端服务器haproxy-1.4.9以后版本
frontend kube-apiserver
bind *:6443
mode tcp
option tcplog
default_backend kube-apiserver
frontend kubesphere-web
bind *:30880
mode http
option httplog
default_backend kubesphere-web
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend kube-apiserver
mode tcp
option tcplog
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server kube-apiserver-1 192.168.3.40:6443 check
server kube-apiserver-2 192.168.3.41:6443 check
server kube-apiserver-3 192.168.3.42:6443 check
backend kubesphere-web
mode http
option httplog
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server kubesphere-web1 192.168.3.40:30880 check
server kubesphere-web2 192.168.3.41:30880 check
server kubesphere-web3 192.168.3.42:30880 check
### 开启cookie会话控制
backend kubesphere-web
mode http
option httplog
balance roundrobin
cookie _S insert nocache
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server kubesphere-web1 192.168.3.40:30880 cookie 40 check
server kubesphere-web2 192.168.3.41:30880 cookie 41 check
server kubesphere-web3 192.168.3.42:30880 cookie 42 check
vim /etc/rsyslog.conf
#由于haproxy的日志是用udp传输的,所以要启用rsyslog的udp监听
Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
#启用级别为local2的设备,并将该设备的所有级别的日志全部输出到/var/log/haproxy.log下
local2.* /var/log/haproxy.log
配置完毕,之后重启rsyslog服务即可