kubernetes二进制部署----多节点部署环境以及负载均衡部署

最新推荐文章于 2024-05-16 22:07:59 发布

这小伙长的不孬

最新推荐文章于 2024-05-16 22:07:59 发布

阅读量290

点赞数 3

分类专栏： Kubernetes 文章标签： kubernetes docker linux 运维负载均衡

本文链接：https://blog.csdn.net/weixin_51622156/article/details/115719155

版权

Kubernetes 专栏收录该内容

7 篇文章 0 订阅

订阅专栏

文章目录

1、master02部署
2、负载均衡部署

先具备单master节点部署环境

1、master02部署

1.1、优先关闭防火墙和selinux服务

1.2、复制kubernetes目录到master02

[root@localhost k8s]# scp -r /opt/kubernetes/ root@192.168.162.20:/opt
The authenticity of host '192.168.195.131 (192.168.195.131)' can't be established.
ECDSA key fingerprint is SHA256:IJ43xXlBWD7qPaL/uFG+4qW4qd7C8xBqUttHiYME8YE.
ECDSA key fingerprint is MD5:cf:3e:dc:e5:89:86:e9:43:38:ee:31:9d:8c:d4:75:9f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.195.131' (ECDSA) to the list of known hosts.
root@192.168.195.131's password: 
token.csv                                                         100%   84    86.1KB/s   00:00    
kube-apiserver                                                    100%  939     1.2MB/s   00:00    
kube-scheduler                                                    100%   94    52.0KB/s   00:00    
kube-controller-manager                                           100%  483   446.5KB/s   00:00    
kube-apiserver                                                    100%  184MB  30.6MB/s   00:06    
kubectl                                                           100%   55MB  32.1MB/s   00:01    
kube-controller-manager                                           100%  155MB  31.1MB/s   00:05    
kube-scheduler                                                    100%   55MB  30.7MB/s   00:01    
ca-key.pem                                                        100% 1679   741.3KB/s   00:00    
ca.pem                                                            100% 1359     1.5MB/s   00:00    
server-key.pem                                                    100% 1675     1.3MB/s   00:00    
server.pem                                                        100% 1643     1.6MB/s   00:00   
//复制master中的三个组件启动脚本kube-apiserver.service                   kube-controller-manager.service        kube-scheduler.service  

[root@localhost k8s]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.162.20:/usr/lib/systemd/system/
root@192.168.195.131's password: 
kube-apiserver.service                                            100%  282   268.1KB/s   00:00    
kube-controller-manager.service                                   100%  317   294.2KB/s   00:00    
kube-scheduler.service                                            100%  281   257.5KB/s   00:00

1.3、master02上操作

1.4、修改配置文件kube-apiserver中的IP

[root@localhost ~]# cd /opt/kubernetes/cfg/
[root@localhost cfg]# vim kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.195.149:2379,https://192.168.195.150:2379,https://192.168.195.151:2379 \
--bind-address=192.168.162.20 \
--secure-port=6443 \
--advertise-address=192.168.162.20 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem  \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

1.5、特别注意：master02一定要有etcd证书

需要拷贝master01上已有的etcd证书给master02使用
[root@localhost k8s]# scp -r /opt/etcd/ root@192.168.162.20:/opt/
root@192.168.195.131's password: 
etcd                                                              100%  523   415.0KB/s   00:00    
etcd                                                              100%   18MB  42.7MB/s   00:00    
etcdctl                                                           100%   15MB  35.2MB/s   00:00    
ca-key.pem                                                        100% 1675   612.1KB/s   00:00    
ca.pem                                                            100% 1265     1.0MB/s   00:00    
server-key.pem                                                    100% 1679     1.7MB/s   00:00    
server.pem                                                        100% 1338     1.7MB/s   00:00

1.6、启动master02中的三个组件服务

[root@localhost cfg]# systemctl start kube-apiserver.service 
[root@localhost cfg]# systemctl start kube-controller-manager.service 
[root@localhost cfg]# systemctl start kube-scheduler.service

1.7、增加环境变量

[root@localhost cfg]# vim /etc/profile
#末尾添加
export PATH=$PATH:/opt/kubernetes/bin/
[root@localhost cfg]# source /etc/profile
[root@localhost cfg]# kubectl get node
NAME              STATUS   ROLES    AGE     VERSION
192.168.195.150   Ready    <none>   2d12h   v1.12.3
192.168.195.151   Ready    <none>   38h     v1.12.3

2、负载均衡部署

2.1、安装nginx服务，把nginx.sh和keepalived.conf脚本拷贝到家目录

lb01 lb02操作

[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0
[root@localhost ~]# vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
[root@localhost ~]# yum install nginx -y
//添加四层转发
[root@localhost ~]# vim /etc/nginx/nginx.conf 

events {
    worker_connections  1024;
}
stream {

   log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
    access_log  /var/log/nginx/k8s-access.log  main;

    upstream k8s-apiserver {
        server 192.168.195.149:6443;
        server 192.168.195.131:6443;
    }
    server {
                listen 6443;
                proxy_pass k8s-apiserver;
    }
    }
http {
[root@localhost ~]# systemctl start nginx

2.2、部署keepalived服务

[root@localhost ~]# yum install keepalived -y

2.3、修改配置文件

[root@localhost ~]# cp keepalived.conf /etc/keepalived/keepalived.conf 
cp：是否覆盖"/etc/keepalived/keepalived.conf"？ yes
//注意：lb01是Mster配置如下：

! Configuration File for keepalived 
 
global_defs { 
   # 接收邮件地址 
   notification_email { 
     acassen@firewall.loc 
     failover@firewall.loc 
     sysadmin@firewall.loc 
   } 
   # 邮件发送地址 
   notification_email_from Alexandre.Cassen@firewall.loc  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER 
} 

vrrp_script check_nginx {
    script "/etc/nginx/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state MASTER 
    interface ens33
    virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 
    priority 100    # 优先级，备服务器设置 90 
    advert_int 1    # 指定VRRP 心跳包通告间隔时间，默认1秒 
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        192.168.195.100/24 
    } 
    track_script {
        check_nginx
    } 
}//注意：lb02是Backup配置如下：
! Configuration File for keepalived 
 
global_defs { 
   # 接收邮件地址 
   notification_email { 
     acassen@firewall.loc 
     failover@firewall.loc 
     sysadmin@firewall.loc 
   } 
   # 邮件发送地址 
   notification_email_from Alexandre.Cassen@firewall.loc  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER 
} 

vrrp_script check_nginx {
    script "/etc/nginx/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state BACKUP 
    interface ens33
    virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 
    priority 90    # 优先级，备服务器设置 90 
    advert_int 1    # 指定VRRP 心跳包通告间隔时间，默认1秒 
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        192.168.195.100/24 
    } 
    track_script {
        check_nginx
    } 
}

[root@localhost ~]# vim /etc/nginx/check_nginx.sh
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")

if [ "$count" -eq 0 ];then
    systemctl stop keepalived
fi
[root@localhost ~]# chmod +x /etc/nginx/check_nginx.sh
[root@localhost ~]# systemctl start keepalived

2.4、查看lb01地址信息

[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:eb:11:2a brd ff:ff:ff:ff:ff:ff
    inet 192.168.195.133/24 brd 192.168.195.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.195.100/24 scope global secondary ens33  //漂移地址在lb01中
       valid_lft forever preferred_lft forever
    inet6 fe80::53ba:daab:3e22:e711/64 scope link 
       valid_lft forever preferred_lft forever

2.5、查看lb02地址信息

[root@localhost nginx]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c9:9d:88 brd ff:ff:ff:ff:ff:ff
    inet 192.168.195.136/24 brd 192.168.195.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::55c0:6788:9feb:550d/64 scope link 
       valid_lft forever preferred_lft forever

//验证地址漂移（lb01中使用pkill nginx，再在lb02中使用ip a 查看）
//恢复操作（在lb01中先启动nginx服务，再启动keepalived服务）
//nginx站点/usr/share/nginx/html

2.6、开始修改node节点配置文件统一VIP（bootstrap.kubeconfig,kubelet.kubeconfig）

[root@localhost cfg]# vim /opt/kubernetes/cfg/bootstrap.kubeconfig
[root@localhost cfg]# vim /opt/kubernetes/cfg/kubelet.kubeconfig
[root@localhost cfg]# vim /opt/kubernetes/cfg/kube-proxy.kubeconfig
//统统修改为VIP
server: https://192.168.162.100:6443
[root@localhost cfg]# systemctl restart kubelet.service 
[root@localhost cfg]# systemctl restart kube-proxy.service 
//替换完成直接自检
[root@localhost cfg]# grep 100 *
bootstrap.kubeconfig:    server: https://192.168.195.100:6443
kubelet.kubeconfig:    server: https://192.168.195.100:6443
kube-proxy.kubeconfig:    server: https://192.168.195.100:6443

2.7、在lb01上查看nginx的k8s日志

[root@localhost ~]# tail /var/log/nginx/k8s-access.log 
192.168.195.150 192.168.195.149:6443 - [05/Feb/2020:12:43:50 +0800] 200 1121
192.168.195.150 192.168.195.131:6443 - [05/Feb/2020:12:43:50 +0800] 200 1120
192.168.195.151 192.168.195.131:6443 - [05/Feb/2020:12:45:38 +0800] 200 1121
192.168.195.151 192.168.195.149:6443 - [05/Feb/2020:12:45:38 +0800] 200 1121

2.8、在master01上操作

测试创建pod

[root@localhost ~]# kubectl run nginx --image=nginx
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created
//查看状态
[root@localhost ~]# kubectl get pods
NAME                    READY   STATUS              RESTARTS   AGE
nginx-dbddb74b8-nf9sk   0/1     ContainerCreating   0          33s   //正在创建中
[root@localhost ~]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-nf9sk   1/1     Running   0          80s  //创建完成，运行中

2.9、注意日志问题

[root@localhost ~]# kubectl logs nginx-dbddb74b8-nf9sk
Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-nf9sk)

[root@localhost ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created
//查看pod网络
[root@localhost ~]# kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE              NOMINATED NODE
nginx-dbddb74b8-nf9sk   1/1     Running   0          11m   172.17.31.3   192.168.195.150   <none>

2.10、在对应网段的node节点上操作可以直接访问

[root@localhost cfg]# curl 172.17.31.3
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
//访问就会产生日志
//回到master01操作
[root@localhost ~]# kubectl logs nginx-dbddb74b8-nf9sk
172.17.31.1 - - [05/Feb/2020:05:08:36 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0"

这小伙长的不孬

关注

3
点赞
踩
0

收藏

觉得还不错? 一键收藏
1
评论
kubernetes二进制部署----多节点部署环境以及负载均衡部署

文章目录master02部署优先关闭防火墙和selinux服务复制kubernetes目录到master02master02上操作修改配置文件kube-apiserver中的IP特别注意：master02一定要有etcd证书启动master02中的三个组件服务增加环境变量负载均衡部署安装nginx服务，把nginx.sh和keepalived.conf脚本拷贝到家目录部署keepalived服务修改配置文件查看lb01地址信息查看lb02地址信息开始修改node节点配置文件统一VIP（bootstrap.k
复制链接

扫一扫