Kubernetes集群部署v1.17.0

最新推荐文章于 2023-05-27 21:32:29 发布

再看我把你吃掉

最新推荐文章于 2023-05-27 21:32:29 发布

阅读量1.4k

点赞数

分类专栏： Kubernetes

本文链接：https://blog.csdn.net/zz_aiytag/article/details/103816076

版权

Kubernetes 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

文章目录

本地环境
环境准备
安装docker
安装kubelet和kubeadm
查看集群中pod和service信息
测试集群
- nginx测试

本地环境

软件	版本
centos	7.4-1708
docker	18.03.0-ce
kubernetes	1.17.0

环境准备

设置hostname及域名解析

各节点分别设置主机名

hostnamectl set-hostname master.node

cat >> /etc/hosts <<EOF
10.XX.XX.52 master.node
10.XX.XX.51 worker1.node
10.XX.XX.50 worker2.node
EOF

设置域名解析

cat >> /etc/resolv.conf <<EOF
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF

关闭防火墙

systemctl stop firewalld
systemctl disable firewalld.service

关闭swap及selinux

swapoff -a

或者修改/etc/fstab，将swap那一项注释掉（需重启，永久禁用）

# /etc/fstab
# Created by anaconda on Fri Dec 20 15:28:07 2019
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=21bbe1f9-7e7b-47db-b13a-e54edecf4342 /                       xfs     defaults        0 0
UUID=45103268-084f-4ad5-ab3f-703edda4456a /boot                   xfs     defaults        0 0
# UUID=af6af595-3cd9-4fdc-9489-689eb64fd8c6 swap                    swap    defaults        0 0

关闭SELINUX

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

master对node节点免密登陆

master节点执行

ssh-keygen -t rsa

分别分发到各个节点

ssh-copy-id -i /root/.ssh/id_rsa.pub 10.XX.XX.52
ssh-copy-id -i /root/.ssh/id_rsa.pub 10.XX.XX.51
ssh-copy-id -i /root/.ssh/id_rsa.pub 10.XX.XX.50

保证时间同步

网桥设置

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

安装docker

参考

https://blog.csdn.net/zz_aiytag/article/details/103660090

安装kubelet和kubeadm

增加下载源

cat <<EOF >  /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF

所有节点执行yum -y install kubelet kubeadm

yum -y install kubelet kubeadm

启动kubelet服务

systemctl start kubelet
systemctl enable kubelet

master初始化

 kubeadm init --apiserver-advertise-address=XX.XX.XX.52 --kubernetes-version=v1.17.0 --pod-network-cidr=10.200.0.0/16 --image-repository registry.aliyuncs.com/google_containers

 kubeadm init --apiserver-advertise-address=XX.XX.XX.52 \
   --kubernetes-version=v1.17.0 \
   --pod-network-cidr=10.200.0.0/16 \
   --image-repository registry.aliyuncs.com/google_containers

[root@master ~]# kubeadm init --apiserver-advertise-address=10.XX.XX.52 --kubernetes-version=v1.17.0 --pod-network-cidr=10.200.0.0/16 --image-repository registry.aliyuncs.com/google_containers
W0103 13:49:18.212117    1928 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0103 13:49:18.212158    1928 validation.go:28] Cannot validate kubelet config - no validator is available
[init] Using Kubernetes version: v1.17.0
[preflight] Running pre-flight checks
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 18.03.0-ce. Latest validated version: 19.03
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [master.node kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.XX.XX.52]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [master.node localhost] and IPs [10.XX.XX.52 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [master.node localhost] and IPs [10.XX.XX.52 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0103 13:56:47.334195    1928 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0103 13:56:47.335496    1928 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 34.001970 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master.node as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node master.node as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 8mmfuc.88q0nmk4fkqlerh3
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.XX.XX.52:6443 --token 8mmfuc.88q0nmk4fkqlerh3 \
    --discovery-token-ca-cert-hash sha256:cb4b0096e35a9e7cbdef3cc3d029e50e01d14f0eae5176c660f5dd32ef9cf8b7

根据上面的输出提示进行操作

[root@worker centos7]# mkdir -p $HOME/.kube
[root@worker centos7]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@worker centos7]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@worker centos7]# cd /root/.kube/
[root@worker .kube]# ll
total 8
-rw------- 1 root root 5453 Dec 31 08:36 config

node节点加入集群

[root@worker1 ~]# kubeadm join 10.XX.XX.52:6443 --token 8mmfuc.88q0nmk4fkqlerh3 \
>     --discovery-token-ca-cert-hash sha256:cb4b0096e35a9e7cbdef3cc3d029e50e01d14f0eae5176c660f5dd32ef9cf8b7
W0103 13:57:49.773475    1156 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 18.03.0-ce. Latest validated version: 19.03
	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

安装网络插件flannel

在没有安装flannel之前，集群状态是这样的

[root@master .kube]# kubectl get nodes
NAME           STATUS     ROLES    AGE    VERSION
master.node    NotReady   master   6m4s   v1.17.0
worker1.node   NotReady   <none>   5m9s   v1.17.0
worker2.node   NotReady   <none>   10s    v1.17.0

[root@master kubernetes]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Fri 2020-01-03 13:57:22 CST; 1h 15min ago
     Docs: https://kubernetes.io/docs/
 Main PID: 3319 (kubelet)
   Memory: 43.2M
   CGroup: /system.slice/kubelet.service
           └─3319 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-image=registry.aliyunc...

Jan 03 15:12:41 master.node kubelet[3319]: W0103 15:12:41.692331    3319 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
Jan 03 15:12:42 master.node kubelet[3319]: E0103 15:12:42.067528    3319 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jan 03 15:12:46 master.node kubelet[3319]: W0103 15:12:46.692518    3319 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
Jan 03 15:12:47 master.node kubelet[3319]: E0103 15:12:47.068386    3319 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jan 03 15:12:51 master.node kubelet[3319]: W0103 15:12:51.692729    3319 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
Jan 03 15:12:52 master.node kubelet[3319]: E0103 15:12:52.069300    3319 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jan 03 15:12:56 master.node kubelet[3319]: W0103 15:12:56.692884    3319 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
Jan 03 15:12:57 master.node kubelet[3319]: E0103 15:12:57.070321    3319 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jan 03 15:13:01 master.node kubelet[3319]: W0103 15:13:01.693006    3319 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
Jan 03 15:13:02 master.node kubelet[3319]: E0103 15:13:02.071537    3319 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

先拉取flannel的镜像，所有的节点都操作

docker pull easzlab/flannel:v0.11.0-amd64
docker tag easzlab/flannel:v0.11.0-amd64 quay.io/coreos/flannel:v0.11.0-amd64

查看docker镜像

[root@master kubernetes]# docker image ls
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
registry.aliyuncs.com/google_containers/kube-proxy                v1.17.0             7d54289267dc        3 weeks ago         116MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.17.0             78c190f736b1        3 weeks ago         94.4MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.17.0             0cae8d5cc64c        3 weeks ago         171MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.17.0             5eb3b7486872        3 weeks ago         161MB
registry.aliyuncs.com/google_containers/coredns                   1.6.5               70f311871ae1        8 weeks ago         41.6MB
registry.aliyuncs.com/google_containers/etcd                      3.4.3-0             303ce5db0e90        2 months ago        288MB
easzlab/flannel                                                   v0.11.0-amd64       ff281650a721        11 months ago       52.6MB
quay.io/coreos/flannel                                            v0.11.0-amd64       ff281650a721        11 months ago       52.6MB
registry.aliyuncs.com/google_containers/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB

下载flannel资源配置清单

wget  https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

修改kube-flannel.yml文件内容

data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.200.0.0/16",  #将默认的IP段改成和master init时指定的一致
      "Backend": {
        "Type": "vxlan"
      }
    }

安装flannel

[root@master opt]# kubectl apply -f kube-flannel.yml 
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created

再查看节点状态

[root@master opt]# kubectl get nodes
NAME           STATUS   ROLES    AGE    VERSION
master.node    Ready    master   102m   v1.17.0
worker1.node   Ready    <none>   101m   v1.17.0
worker2.node   Ready    <none>   96m    v1.17.0

查看集群中pod和service信息

[root@master opt]# kubectl get pods,svc -n kube-system -o wide
NAME                                      READY   STATUS    RESTARTS   AGE    IP              NODE           NOMINATED NODE   READINESS GATES
pod/coredns-9d85f5447-bpkgg               1/1     Running   0          114m   10.200.0.4      master.node    <none>           <none>
pod/coredns-9d85f5447-shx4v               1/1     Running   0          114m   10.200.0.2      master.node    <none>           <none>
pod/etcd-master.node                      1/1     Running   0          114m   10.180.249.52   master.node    <none>           <none>
pod/kube-apiserver-master.node            1/1     Running   0          114m   10.180.249.52   master.node    <none>           <none>
pod/kube-controller-manager-master.node   1/1     Running   0          114m   10.180.249.52   master.node    <none>           <none>
pod/kube-flannel-ds-amd64-cgh52           1/1     Running   0          12m    10.180.249.52   master.node    <none>           <none>
pod/kube-flannel-ds-amd64-cnmnj           1/1     Running   0          12m    10.180.249.50   worker2.node   <none>           <none>
pod/kube-flannel-ds-amd64-mv4vw           1/1     Running   0          12m    10.180.249.51   worker1.node   <none>           <none>
pod/kube-proxy-42hpc                      1/1     Running   0          114m   10.180.249.52   master.node    <none>           <none>
pod/kube-proxy-qgfwt                      1/1     Running   0          108m   10.180.249.50   worker2.node   <none>           <none>
pod/kube-proxy-whh5j                      1/1     Running   0          113m   10.180.249.51   worker1.node   <none>           <none>
pod/kube-scheduler-master.node            1/1     Running   0          114m   10.180.249.52   master.node    <none>           <none>

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE    SELECTOR
service/kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   114m   k8s-app=kube-dns

测试集群

nginx测试

[root@master opt]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
[root@master opt]# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed

[root@master opt]# kubectl get pods,svc --all-namespaces -o wide
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE    IP              NODE           NOMINATED NODE   READINESS GATES
default       pod/nginx-86c57db685-2klmf                1/1     Running   0          15m    10.200.2.2      worker2.node   <none>           <none>
kube-system   pod/coredns-9d85f5447-bpkgg               1/1     Running   0          132m   10.200.0.4      master.node    <none>           <none>
kube-system   pod/coredns-9d85f5447-shx4v               1/1     Running   0          132m   10.200.0.2      master.node    <none>           <none>
kube-system   pod/etcd-master.node                      1/1     Running   0          132m   10.180.249.52   master.node    <none>           <none>
kube-system   pod/kube-apiserver-master.node            1/1     Running   0          132m   10.180.249.52   master.node    <none>           <none>
kube-system   pod/kube-controller-manager-master.node   1/1     Running   0          132m   10.180.249.52   master.node    <none>           <none>
kube-system   pod/kube-flannel-ds-amd64-cgh52           1/1     Running   0          30m    10.180.249.52   master.node    <none>           <none>
kube-system   pod/kube-flannel-ds-amd64-cnmnj           1/1     Running   0          30m    10.180.249.50   worker2.node   <none>           <none>
kube-system   pod/kube-flannel-ds-amd64-mv4vw           1/1     Running   0          30m    10.180.249.51   worker1.node   <none>           <none>
kube-system   pod/kube-proxy-42hpc                      1/1     Running   0          132m   10.180.249.52   master.node    <none>           <none>
kube-system   pod/kube-proxy-qgfwt                      1/1     Running   0          126m   10.180.249.50   worker2.node   <none>           <none>
kube-system   pod/kube-proxy-whh5j                      1/1     Running   0          131m   10.180.249.51   worker1.node   <none>           <none>
kube-system   pod/kube-scheduler-master.node            1/1     Running   0          132m   10.180.249.52   master.node    <none>           <none>

NAMESPACE     NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE    SELECTOR
default       service/kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP                  132m   <none>
default       service/nginx        NodePort    10.96.84.231   <none>        80:32751/TCP             13m    app=nginx
kube-system   service/kube-dns     ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP   132m   k8s-app=kube-dns

只有一个pod，浏览器打开worker2节点的IP:32751可以看到nginx欢迎信息
修改pod副本数量

kubectl scale deployment/nginx --replicas=2

创建过程，需要等待一会

[root@master opt]# kubectl get pods,svc --all-namespaces -o wide
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE     IP              NODE           NOMINATED NODE   READINESS GATES
default       pod/nginx-86c57db685-2klmf                1/1     Running   0          37m     10.200.2.2      worker2.node   <none>           <none>
default       pod/nginx-86c57db685-b4t2h                1/1     Running   0          2m46s   10.200.1.2      worker1.node   <none>           <none>

现在有两个pod在两个节点上，浏览器测试对应节点IP:32751可以看到nginx欢迎信息

再看我把你吃掉

关注

0
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
Kubernetes集群部署v1.17.0

文章目录本地环境环境准备设置hostname及域名解析关闭防火墙关闭swap及selinuxmaster对node节点免密登陆保证时间同步网桥设置安装docker安装kubelet和kubeadmmaster初始化node节点加入集群安装网络插件flannel查看集群中pod和service信息测试集群nginx测试本地环境软件版本centos7.4-1708doc...
复制链接

扫一扫