k8s本地集群搭建(v1.21.2)

最新推荐文章于 2024-09-06 15:17:27 发布

zhaoyunxing92

最新推荐文章于 2024-09-06 15:17:27 发布

阅读量472

点赞数

本文链接：https://blog.csdn.net/qq_29553063/article/details/118612511

版权

k8s前期准备工作

我这里主要使用docker-case构建本地k8s(v1.21.2)集群的

软件准备

vagrant 2.2.16 跨平台虚拟化工具
virtualbox 开源免费的虚拟机
k8s version v1.21.2

vagrant环境说明

需要使用root用户进去开始安装，不然会遇到很多问题

节点	连接方式	密码
master	ssh root@127.0.0.1 -p 2200	vagrant
node1	ssh root@127.0.0.1 -p 2201	vagrant
node2	ssh root@127.0.0.1 -p 2202	vagrant

配置过程

先创建三个虚拟机

进入目录执行下面命令就可以了docker-case/k8s

vagrant up

修改images为阿里云

查看需要的版本

kubeadm config images list

master开始安装

使用阿里云镜像镜像加速，coredns:v1.8.0由于tag问题导致不能下载所以先下载coredns:v1.8.0

查看需要安装软件列表

kubeadm config images list

这里主要看下coredns版本

k8s.gcr.io/kube-apiserver:v1.21.2
k8s.gcr.io/kube-controller-manager:v1.21.2
k8s.gcr.io/kube-scheduler:v1.21.2
k8s.gcr.io/kube-proxy:v1.21.2
k8s.gcr.io/pause:3.4.1
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns/coredns:v1.8.0

下载coredns:1.8.0

docker pull registry.aliyuncs.com/google_containers/coredns:1.8.0

替换tag

docker tag registry.aliyuncs.com/google_containers/coredns:1.8.0 registry.aliyuncs.com/google_containers/coredns:v1.8.0

删除旧的

docker rmi registry.aliyuncs.com/google_containers/coredns:1.8.0

kubeadm init

kubeadm init \
 --apiserver-advertise-address=192.168.56.200 \
 --image-repository registry.aliyuncs.com/google_containers \
 --kubernetes-version v1.21.2 \
 --pod-network-cidr=10.244.0.0/16

初始化kube

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
sh -c "echo 'export KUBECONFIG=/etc/kubernetes/admin.conf' >> /etc/profile"
source /etc/profile

安装网络插件flannle

由于我们上面使用的ip(192.168.56.200)不在eth0网卡上需要修改网卡可以通过ip add查看网卡

1. 先下载

可以直接用kube-flannel.yml

curl https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml > kube-flannel.yml

2.开始编辑`vi kube-flannel.yml`

    181       containers:
    182       - name: kube-flannel
    183         image: quay.io/coreos/flannel:v0.14.0
    184         command:
    185         - /opt/bin/flanneld
    186         args:
    187         - --ip-masq
    188         - --kube-subnet-mgr
    189         - --iface=eth1 # 这个地方指定网卡

3.apply flannel

[root@k8s-master ~]# kubectl apply -f kube-flannel.yml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

4.验证结果

先获取pods然后查看一个pod的日志即可

[root@k8s-master ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-59d64cd4d4-2d46n             1/1     Running   0          19m
kube-system   coredns-59d64cd4d4-jvbvs             1/1     Running   0          19m
kube-system   etcd-k8s-master                      1/1     Running   0          20m
kube-system   kube-apiserver-k8s-master            1/1     Running   0          20m
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          15m
kube-system   kube-flannel-ds-hvsmm                1/1     Running   0          95s
kube-system   kube-flannel-ds-lhdpp                1/1     Running   0          95s
kube-system   kube-flannel-ds-nbxl4                1/1     Running   0          95s
kube-system   kube-proxy-6gj8p                     1/1     Running   0          18m
kube-system   kube-proxy-jf8v6                     1/1     Running   0          18m
kube-system   kube-proxy-srx5t                     1/1     Running   0          19m
kube-system   kube-scheduler-k8s-master            1/1     Running   0          15m

由日志可以看到Using interface with name eth1 and address 192.168.56.200

[root@k8s-master ~]# kubectl logs -n kube-system kube-flannel-ds-nbxl4
I0709 08:28:28.858390       1 main.go:533] Using interface with name eth1 and address 192.168.56.200
I0709 08:28:28.859608       1 main.go:550] Defaulting external address to interface address (192.168.56.200)
W0709 08:28:29.040955       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.

使用ipvs负载均衡

啥是ipvs 可以看看

修改配置

执行kubectl edit configmap -n kube-system kube-proxy

     28     iptables:
     29       masqueradeAll: false
     30       masqueradeBit: null
     31       minSyncPeriod: 0s
     32       syncPeriod: 0s
     33     ipvs:
     34       excludeCIDRs: null
     35       minSyncPeriod: 0s
     36       scheduler: ""
     37       strictARP: false
     38       syncPeriod: 0s
     39       tcpFinTimeout: 0s
     40       tcpTimeout: 0s
     41       udpTimeout: 0s
     42     kind: KubeProxyConfiguration
     43     metricsBindAddress: ""
     44     mode: "ipvs" # 这里指定用ipvs
     45     nodePortAddresses: null
     46     oomScoreAdj: null

杀死所有的 `kube-proxy` pod

[root@k8s-master ~]# kubectl get pod -n kube-system |grep kube-proxy
kube-proxy-6gj8p                     1/1     Running   0          149m
kube-proxy-jf8v6                     1/1     Running   0          150m
kube-proxy-srx5t                     1/1     Running   0          150m
[root@k8s-master ~]# kubectl delete  pod -n kube-system kube-proxy-6gj8p kube-proxy-jf8v6 kube-proxy-srx5t
pod "kube-proxy-6gj8p" deleted
pod "kube-proxy-jf8v6" deleted
pod "kube-proxy-srx5t" deleted

验证结果

Using ipvs Proxier.

[root@k8s-master ~]# kubectl get pod -n kube-system |grep kube-proxy
kube-proxy-4fvxf                     1/1     Running   0          2m41s
kube-proxy-j79lj                     1/1     Running   0          2m32s
kube-proxy-l4vjz                     1/1     Running   0          2m41s
[root@k8s-master ~]# kubectl logs -n kube-system kube-proxy-4fvxf
Error from server (NotFound): the server could not find the requested resource ( pods/log kube-proxy-4fvxf)
[root@k8s-master ~]# kubectl logs -n kube-system kube-proxy-j79lj
I0709 10:43:06.511908       1 node.go:172] Successfully retrieved node IP: 10.0.2.15
I0709 10:43:06.511989       1 server_others.go:140] Detected node IP 10.0.2.15
I0709 10:43:06.551606       1 server_others.go:206] kube-proxy running in dual-stack mode, IPv4-primary
I0709 10:43:06.551773       1 server_others.go:274] Using ipvs Proxier.
I0709 10:43:06.551794       1 server_others.go:276] creating dualStackProxier for ipvs.
W0709 10:43:06.551816       1 server_others.go:512] detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6

kubeadm token 重新设置

这一步不是必须，如果上面join忘记了可以重新生成

kubeadm token create --print-join-command

可能遇到的问题

scheduler和controller-manager端口起不来

[root@k8s-master ~]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS      MESSAGE                                                                                       ERROR
scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused
controller-manager   Unhealthy   Get "http://127.0.0.1:10252/healthz": dial tcp 127.0.0.1:10252: connect: connection refused
etcd-0               Healthy     {"health":"true"}

解决

主要问题是--port=0导致，注释掉就可以

vi /etc/kubernetes/manifests/kube-scheduler.yaml 注释端口

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-scheduler
    tier: control-plane
  name: kube-scheduler
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-scheduler
    - --authentication-kubeconfig=/etc/kubernetes/scheduler.conf
    - --authorization-kubeconfig=/etc/kubernetes/scheduler.conf
    - --bind-address=127.0.0.1
    - --kubeconfig=/etc/kubernetes/scheduler.conf
    - --leader-elect=true
#    - --port=0  # 注释端口
    image: registry.aliyuncs.com/google_containers/kube-scheduler:v1.21.2
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 127.0.0.1
        path: /healthz
        port: 10259
        scheme: HTTPS
.....

vi /etc/kubernetes/manifests/kube-controller-manager.yaml 同样注释端口

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-controller-manager
    tier: control-plane
  name: kube-controller-manager
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-controller-manager
    - --allocate-node-cidrs=true
    - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
    - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
    - --bind-address=127.0.0.1
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --cluster-cidr=10.24.0.0/16
    - --cluster-name=kubernetes
    - --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
    - --cluster-signing-key-file=/etc/kubernetes/pki/ca.key
    - --controllers=*,bootstrapsigner,tokencleaner
    - --kubeconfig=/etc/kubernetes/controller-manager.conf
    - --leader-elect=true
#    - --port=0
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --root-ca-file=/etc/kubernetes/pki/ca.crt
    - --service-account-private-key-file=/etc/kubernetes/pki/sa.key
    - --service-cluster-ip-range=10.96.0.0/12
    - --use-service-account-credentials=true
    image: registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.2
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 127.0.0.1
        path: /healthz
        port: 10257
        scheme: HTTPS
.....

重新启动

systemctl restart kubelet.service

kubeadm join的时候提示没有关闭bridge-nf-call-iptables

error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1

解决

sudo sh -c " echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables"

node无法加入master节点

之前我使用的ip是192.168.56.100跟VirualBox的DHCP服务冲突了选择端口的时候需要注意下ip选择范围

在这里插入图片描述