问题背景:
本人在部署k8s高可用集群的时候,选择了三台阿里云的服务器,做为集群的三个master,在三个机器上搞完了keepalived和haproxy的高可用后,发现彼此之间网络不通了,后来了解到阿里云有自己的slb,不支持keepalived方式,遂使用了阿里云的slb,待负载均衡创建好后,又发现了不支持回环。
那么作为k8s的master节点,去访问vip,vip负载到master,其本身必然会形成回环:
而阿里云的回环机制很"单纯",我们只要在中间加一个"第三者"即可解决,但是nginx不能在master节点上。
如图:
步骤:
使用docker部署nginx,实现tcp端口转发
docker pull nginx
vim /home/work/online/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
stream {
upstream k8s-vip {
server 172.16.198.225:6443;
server 172.16.198.226:6443;
server 172.16.198.227:6443;
}
server {
listen 8443;
proxy_pass k8s-vip;
}
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access1.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
# docker run启动nginx
docker run --name nginx -p 80:80 -p 8443:8443 -v /home/work/online/nginx/nginx.conf:/etc/nginx/nginx.conf -v /home/work/online/nginx/log:/var/log/nginx -v -d nginx
启动完nginx别忘了在阿里云的slb上做一下vip到nginx的负载就可以了,nginx的启动数量可以根据情况而定。