centos7 使用kubeadm 快速部署 kubernetes 国内源

最新推荐文章于 2024-09-11 18:00:58 发布
最新推荐文章于 2024-09-11 18:00:58 发布
阅读量1.5k 收藏 3
点赞数
文章标签： 运维 嵌入式 开发工具
原文链接：http://www.cnblogs.com/qingfeng2010/p/10540832.html
版权
本文详细介绍了使用kubeadm在CentOS 7环境下部署Kubernetes集群的过程，包括环境准备、Kubernetes组件安装、网络配置、节点管理及常见问题解决。通过实际案例，展示了如何创建部署和服务，以及如何在多节点环境中进行资源调度。
摘要由CSDN通过智能技术生成

前言

搭建kubernetes时看文档以及资料走了很多弯路，so 整理了最后成功安装的过程已做记录。网上的搭建文章总是少一些步骤，想本人这样的小白总是部署不成功(^_^)。

准备两台或两台以上的虚拟机，系统centos7, 本文只准备了两个虚拟机(电脑风扇已转的飞起)。

多注意红色加粗的 代码以及文字 ------qingfeng

我开始整理这个文章的时候使用的是 kubernetes 1.13, 但是当我发布的时候阿里云的源已经更新到了 kubernetes 1.14 所以我想这个文章笔记对用kubeadm 安装 kubernetes 都会有帮助

过程中碰到的问题可以先查一下文章最后看看有没有碰到同样的问题

基础环境准备

两台机器信息

10.211.55.6   k8s-master

10.211.55.7   k8s-node

#设置hostname 的方法 
hostnamectl set-hostname k8s-master  #在 10.211.55.6 上执行
hostnamectl set-hostname k8s-node    #在 10.211.55.7 上执行
hostnamectl --static #查看设置结果

 

所有操作无特殊说明都需要在所有节点(k8s-master 和 k8s-node)上执行

 

关闭防火墙 :: 如果不想启用防火墙，设置可以参考这里看一下kubernetes需要开放的端口 https://kubernetes.io/docs/setup/independent/install-kubeadm/#check-required-ports

systemctl stop firewalld.service
systemctl disable firewalld.service
yum upgrade

 关闭swap :: kubernetes1.8开始不关闭swap无法启动

#去掉 /etc/fstab 里面这一行 /dev/mapper/centos-swap swap                    swap    defaults        0 0
swapoff -a
cp /etc/fstab /etc/fstab_bak
cat /etc/fstab_bak |grep -v swap > /etc/fstab
cat /etc/fstab

修改iptables参数 :: RHEL / CentOS 7上的一些用户报告了由于iptables被绕过而导致流量路由不正确的问题。创建/etc/sysctl.d/k8s.conf文件，添加如下内容：

cat <<EOF >  /etc/sysctl.d/k8s.conf
vm.swappiness = 0
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF


#使配置生效
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf

加载ipvs模块

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF

#这条命令有点长
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

 

安装docker :: 注意docker版本， 现在最高18.06版本做了验证

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum makecache fast
yum install -y --setopt=obsoletes=0 docker-ce-18.06.1.ce-3.el7
systemctl start docker
systemctl enable docker

#查看docker版本号
docker -v
Docker version 18.06.1-ce, build e68fc7a

用kubeadm 部署 kubernetes

安装kubeadm, kubelet  注意:: yum install 安装的时候一定要看一下kubernetes的版本号后面kubeadm init 的时候需要用到

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

#安装   注意：：这里一定要看一下版本号，因为 Kubeadm init 的时候 填写的版本号不能低于kuberenete版本
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
#注 如果需要指定版本 用下面的命令   kubelet-<version>
yum install kubelet-1.14.1 kubeadm-1.14.1 kubectl-1.14.1 --disableexcludes=kubernetes

#启动 kubelet 
systemctl enable kubelet.service && systemctl start kubelet.service

 启动kubelet.service之后 我们查看一下kubelet状态是未启动状态，查看原因发现是 “/var/lib/kubelet/config.yaml”文件不存在，这里可以暂时先不用处理，当kubeadm init 之后会创建此文件

#查看 kubelet 状态
[root@centos2 ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: activating (auto-restart) (Result: exit-code) since 日 2019-03-31 16:18:55 CST; 7s ago
     Docs: https://kubernetes.io/docs/
  Process: 4564 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)
 Main PID: 4564 (code=exited, status=255)

3月 31 16:18:55 k8s-node systemd[1]: Unit kubelet.service entered failed state.
3月 31 16:18:55 k8s-node systemd[1]: kubelet.service failed.
[root@centos2 ~]#


#查看出错信息
[root@centos2 ~]# journalctl -xefu kubelet
3月 31 16:19:46 k8s-node systemd[1]: kubelet.service holdoff time over, scheduling restart.
3月 31 16:19:46 k8s-node systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit kubelet.service has finished shutting down.
3月 31 16:19:46 k8s-node systemd[1]: Started kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit kubelet.service has finished starting up.
--
-- The start-up result is done.
3月 31 16:19:46 k8s-node kubelet[4611]: F0331 16:19:46.989588    4611 server.go:193] failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/kubelet/config.yaml", error: open /var/lib/kubelet/config.yaml: no such file or directory
3月 31 16:19:46 k8s-node systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
3月 31 16:19:46 k8s-node systemd[1]: Unit kubelet.service entered failed state.
3月 31 16:19:46 k8s-node systemd[1]: kubelet.service failed.

 

 我们在 k8s-master上用kubeadm ini初始化kubernetes ：： 注意：：这里的kubernetes-version 一定要和上面安装的版本号一致 否则会报错，报错信息可以参考文章后面错误集锦

#只在 k8s-master上执行   node节点不执行
kubeadm init \
--apiserver-advertise-address=10.211.55.6 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.13.1 \
--pod-network-cidr=10.244.0.0/16

 

--apiserver-advertise-addres :: 填写 k8s-master ip

--image-repository :: 镜像地址

--kubernetes-version :: 关闭版本探测，因为它的默认值是stable-1，会从https://storage.googleapis.com/kubernetes-release/release/stable-1.txt下载最新的版本号，指定版本跳过网络请求，再次强调一定要和Kubernetes版本号一致

 

kubeadm init 初始化信息,  我们看一下初始化过程发现自动创建了 "/var/lib/kubelet/config.yaml" 这个文件

[init] Using Kubernetes version: v1.13.1
[preflight] Running pre-flight checks
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [centos kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.211.55.6]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [centos localhost] and IPs [10.211.55.6 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [centos localhost] and IPs [10.211.55.6 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 19.507714 seconds
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.13" in namespace kube-system with the configuration for the kubelets in the cluster
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "centos" as an annotation
[mark-control-plane] Marking the node centos as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node centos as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: sfaff2.iet15233unw5jzql
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

#======这里是用时再使用集群之前需要执行的操作------qingfeng
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:
  #=====这是增加节点的方法 token过期 请参考问题集锦------qingfeng
  kubeadm join 10.211.55.6:6443 --token sfaff2.iet15233unw5jzql --discovery-token-ca-cert-hash sha256:f798c5be53416ca3b5c7475ee0a4199eb26f9e31ee7106699729c0660a70f8d7

[root@centos ~]#

初始化成功后会提示在使用之前需要再配置一下，配置方法已经给出，另外会生成一个临时token以及增加节点的方法

#普通用户要使用k8s 需要执行下面操作
  
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config


#如果是root 可以直接执行
export KUBECONFIG=/etc/kubernetes/admin.conf

# 以上两个二选一即可，这里我是直接用的root 所以直接执行
export KUBECONFIG=/etc/kubernetes/admin.conf

 现在我们查看一下 kubelet 的状态 已经是 running 状态 ，启动成功

[root@k8s-master ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since 日 2019-03-31 16:11:57 CST; 26min ago
     Docs: https://kubernetes.io/docs/
 Main PID: 32083 (kubelet)
    Tasks: 16
   Memory: 39.6M
   CGroup: /system.slice/kubelet.service
           └─32083 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=systemd --network-plugin=cni --pod-infra-...

3月 31 16:38:28 k8s-master kubelet[32083]: W0331 16:38:28.028997   32083 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
3月 31 16:38:28 k8s-master kubelet[32083]: E0331 16:38:28.752039   32083 kubelet.go:2170] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not read...fig uninitialized
3月 31 16:38:33 k8s-master kubelet[32083]: W0331 16:38:33.029684   32083 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
3月 31 16:38:33 k8s-master kubelet[32083]: E0331 16:38:33.754045   32083 kubelet.go:2170] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not read...fig uninitialized
3月 31 16:38:38 k8s-master kubelet[32083]: W0331 16:38:38.030077   32083 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
3月 31 16:38:38 k8s-master kubelet[32083]: E0331 16:38:38.756061   32083 kubelet.go:2170] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not read...fig uninitialized
3月 31 16:38:43 k8s-master kubelet[32083]: W0331 16:38:43.030827   32083 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
3月 31 16:38:43 k8s-master kubelet[32083]: E0331 16:38:43.757292   32083 kubelet.go:2170] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not read...fig uninitialized
3月 31 16:38:48 k8s-master kubelet[32083]: W0331 16:38:48.031403   32083 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
3月 31 16:38:48 k8s-master kubelet[32083]: E0331 16:38:48.758876   32083 kubelet.go:2170] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not read...fig uninitialized
Hint: Some lines were ellipsized, use -l to show in full.

 

查看状态 ::确认每个 组件都是  Healthy 状态

[root@centos ~]kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health": "true"}

查看node状态  da

[root@centos ~]kubectl get node
NAME     STATUS     ROLES    AGE   VERSION
centos   NotReady   master   11m   v1.13.4

安装port Network( flannel )  :: k8s cluster 工作 必须安装pod网络，否则pod之间无法通信，k8s支持多种方案，这里选择flannel

[root@centos ~]kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
podsecuritypolicy.extensions/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created
[root@centos ~]

检查pod状态，需要确保当前Pod 都是 running

[root@centos ~]kubectl get pod --all-namespaces -o wide
NAMESPACE     NAME                             READY   STATUS    RESTARTS   AGE     IP            NODE     NOMINATED NODE   READINESS GATES
kube-system   coredns-78d4cf999f-6b5wq         1/1     Running   0          5h1m    10.244.0.2    centos   <none>           <none>
kube-system   coredns-78d4cf999f-clhkc         1/1     Running   0          5h1m    10.244.0.3    centos   <none>           <none>
kube-system   etcd-centos                      1/1     Running   0          5h      10.211.55.6   centos   <none>           <none>
kube-system   kube-apiserver-centos            1/1     Running   0          5h      10.211.55.6   centos   <none>           <none>
kube-system   kube-controller-manager-centos   1/1     Running   0          5h      10.211.55.6   centos   <none>           <none>
kube-system   kube-flannel-ds-amd64-lnp55      1/1     Running   0          3m41s   10.211.55.6   centos   <none>           <none>
kube-system   kube-proxy-xsnr8                 1/1     Running   0          5h1m    10.211.55.6   centos   <none>           <none>
kube-system   kube-scheduler-centos            1/1     Running   0          5h      10.211.55.6   centos   <none>           <none>
[root@centos ~]

再次查看node状态； pod状态变为 Ready

[root@centos ~]kubectl get nodes
NAME     STATUS   ROLES    AGE    VERSION
centos   Ready    master   5h2m   v1.13.4
[root@centos ~]

 

至此k8s就安装完成了

  

安装完了赶快体验一下，写个简单的 deployment 测试一下

创建一个 nginx-deployment.yaml  内容如下

[root@k8s-master testnginx]# cat nginx-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: qf-test-nginx
  #namespace: qingfeng-deve
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: qingfenglian/test_nginx
        ports:
        - containerPort: 80

创建 svc, pod, 查看pod状态 发现到这里发现  pod 状态一直是 Pending

[root@k8s-master ~]# mkdir -p k8s/testnginx
[root@k8s-master ~]# cd k8s/testnginx/
[root@k8s-master testnginx]# vim nginx-deployment.yaml
[root@k8s-master testnginx]# vim nginx-deployment.yaml
[root@k8s-master testnginx]# kubectl create -f nginx-deployment.yaml
deployment.extensions/qf-test-nginx created
[root@k8s-master testnginx]# kubectl get svc,pod
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   19m

NAME                                 READY   STATUS    RESTARTS   AGE
pod/qf-test-nginx-56db997f77-gkvcz   0/1     Pending   0          8s

原因是：k8s-master 这台机器不允许创建pod ,想要在k8s-master上创建pod ,实现k8s单点部署可以执行下面操作

#先查看一下 k8s-master这台机器是否允许创建Pod，发现是 NoSchedule
[root@k8s-master testnginx]# kubectl describe node k8s-master | grep Taint
Taints:             node-role.kubernetes.io/master:NoSchedule

#现在允许 k8s-master创建 pod
[root@k8s-master testnginx]# kubectl taint nodes k8s-master node-role.kubernetes.io/master-
node/k8s-master untainted

#我们再来查看一下是否允许创建pod
[root@k8s-master testnginx]# kubectl describe node k8s-master | grep Taint
Taints:             <none>



########--------------分割线------------#########
#可能有的小伙伴测试完成后还想恢复到 k8s-master上禁止创建Pod,可以执行下面操作
kubectl taint nodes k8s-master node-role.kubernetes.io/master=:NoSchedule

当允许k8s-master允许创建pod 之后，我们再来看一下pod状态

[root@k8s-master testnginx]# kubectl describe node k8s-master | grep Taint
Taints:             node-role.kubernetes.io/master:NoSchedule
[root@k8s-master testnginx]# kubectl taint nodes k8s-master node-role.kubernetes.io/master-
node/k8s-master untainted
[root@k8s-master testnginx]# kubectl describe node k8s-master | grep Taint
Taints:             <none>
[root@k8s-master testnginx]# kubectl get pod
NAME                             READY   STATUS    RESTARTS   AGE
qf-test-nginx-56db997f77-gkvcz   1/1     Running   0          8m50s    ----------------这里可以发现pod 已经启动

 

增加node节点 

执行 kubeadm join 

[root@k8s-node ~]# kubeadm join 10.211.55.6:6443 --token uf2c4g.n7ibf1g8gxbkqz2z \
>     --discovery-token-ca-cert-hash sha256:f01892c96cee8d02c373e34bed3a45c8f3f9888fdd19767e706ec09e8fb9c893
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Hostname]: hostname "k8s-node" could not be reached
    [WARNING Hostname]: hostname "k8s-node": lookup k8s-node on 10.211.55.1:53: no such host
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.14" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

现在可以 用 kubectl get node 查看 有多少节点了  , 如果想在node节点上使用kubectl 命令需要把 k8s-master 上 /etc/kubernetes/admin.conf 文件copy到几点机器上并使用 export KUBECONFIG=/etc/kubernetes/admin.conf, 这个在初始化的时候已经提到,, 可以用scp 命令拷贝

[root@k8s-master testnginx]# kubectl get node
NAME         STATUS     ROLES    AGE     VERSION
k8s-master   Ready      master   37m     v1.14.0
k8s-node     NotReady   <none>   2m55s   v1.14.0

copy admin.conf 到 节点机器:: 在节点机器上执行下面命令

[root@k8s-node ~]# kubectl get node                -------------节点 使用kubectl
The connection to the server localhost:8080 was refused - did you specify the right host or port? ----这是报错
[root@k8s-node ~]# scp root@10.211.55.6:/etc/kubernetes/admin.conf /etc/kubernetes/admin.conf    -----从k8s-master 把admin.conf文件copy到当前节点机器上
The authenticity of host '10.211.55.6 (10.211.55.6)' can't be established.
ECDSA key fingerprint is SHA256:ijx7s49ok7H8PMRY0tVKn7Be06G0OjArv/DpCNtHoIw.
ECDSA key fingerprint is MD5:89:68:de:2f:fe:ca:3f:26:e2:28:30:87:2b:21:e9:3d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.211.55.6' (ECDSA) to the list of known hosts.
root@10.211.55.6's password:
admin.conf                                                                                           100% 5451     5.4MB/s   00:00
[root@k8s-node ~]# export KUBECONFIG=/etc/kubernetes/admin.conf      --------导入
[root@k8s-node ~]# kubectl get node    ------------再次使用kubectl 查看 node 
NAME         STATUS   ROLES    AGE     VERSION
k8s-master   Ready    master   43m     v1.14.0
k8s-node     Ready    <none>   8m15s   v1.14.0

 

 

删除node

删除节点之后，节点想再次加入到集群中 需要先执行  kubeadm reset , 之后再执行 kubeadm join

[root@k8s-master testnginx]# kubectl delete node k8s-node    ---k8s-node节点名称，当然不只这一种删除pod的方法，我这里不一一列出了

 

增加节点时token过期，重新生成token的方法, 直接上命令了

[root@k8s-master testnginx]# kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
uf2c4g.n7ibf1g8gxbkqz2z   23h       2019-04-03T15:28:40+08:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
[root@k8s-master testnginx]# kubeadm token create
w0r09e.e5olwz1rlhwvgo9p
[root@k8s-master testnginx]# kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
uf2c4g.n7ibf1g8gxbkqz2z   23h       2019-04-03T15:28:40+08:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
w0r09e.e5olwz1rlhwvgo9p   23h       2019-04-03T16:19:56+08:00   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token
[root@k8s-master testnginx]#

 

由于我们允许k8s-master创建pod ,so现在我们有两个节点了,我们把刚才的 nginx-deployment.yaml 中pod个数改成2 看看结果

#nginx-deployment 修改之后的内容，，其实 只改了 spec.replicas 由1改成 2 
[root@k8s-master testnginx]# cat nginx-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: qf-test-nginx
  #namespace: qingfeng-deve
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: qingfenglian/test_nginx
        ports:
        - containerPort: 80


#重新部署
[root@k8s-master testnginx]# kubectl apply -f nginx-deployment.yaml
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
deployment.extensions/qf-test-nginx configured

#查看pod 状态 发现已经是2个pod的，但是一个还没起来，稍等一下
[root@k8s-master testnginx]# kubectl get pod -o wide
NAME                             READY   STATUS              RESTARTS   AGE   IP           NODE         NOMINATED NODE   READINESS GATES
qf-test-nginx-56db997f77-gkvcz   1/1     Running             0          35m   10.244.0.6   k8s-master   <none>           <none>
qf-test-nginx-56db997f77-tx4wk   0/1     ContainerCreating   0          12s   <none>       k8s-node     <none>           <none>

#这里再看 两个Pod都已经启动成功，，注意看 NODE 列 发现是在不同的NODE上面
[root@k8s-master testnginx]# kubectl get pod -o wide
NAME                             READY   STATUS    RESTARTS   AGE    IP           NODE         NOMINATED NODE   READINESS GATES
qf-test-nginx-56db997f77-gkvcz   1/1     Running   0          37m    10.244.0.6   k8s-master   <none>           <none>
qf-test-nginx-56db997f77-tx4wk   1/1     Running   0          107s   10.244.3.2   k8s-node     <none>           <none>
[root@k8s-master testnginx]#

 

 好了，这次记录就到此结束了。

 

遇到的问题

问题1：kubernetes源的问题

描述：开始是想通过命令行代理的方式解决源的问题，再实际使用中发现在 kubeadm init 的时候会报找不到 k8s-master主机

解决：用aliyum的原

问题2：k8s启动不成功之 docker版本问题

解决：写这篇文章的时候k8s对docker验证最高到 18.06，但此时docker版本已经到了18.09,开始本着测试体验最新版本结果...

其他错误
报错 kubectl get pod  提示 ：：The connection to the server localhost:8080 was refused - did you specify the right host or port?
解决： 将主节点（master节点）中的【/etc/kubernetes/admin.conf】文件拷贝到从节点相同目录下
 
报错：the kubelet version is higher than the control plane version. This is not a supported version skew and may lead to a malfunctional cluster. Kubelet version: "1.14.0" Control plane version: "1.13.1
解决：注意安装过程中红色字体部分，已经提示在kubeadm init 初始化的时候kubernetes的版本
 
报错: [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
解决: 
[root@k8s-master ~]# docker info
Containers: 17
 Running: 16
 Paused: 0
 Stopped: 1
Images: 8
Server Version: 18.06.1-ce
Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd      ######------主要看这里，没有修改之前应该是 cgroup


#修改 docker文件驱动, 修改保存后 重新启动 docker  , docker重新启动方法(systemctl restart docker)

#修改或者创建
vim /etc/docker/daemon.json

#添加如下内容 --这行就不要添加了这只是个注释(^_^)------qingfeng

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

 

阿里云专有网络etcd启动失败

如果是阿里云专有网络的ecs  apiserver-advertise-address的地址应该是私有ip地址。

 

官方文档:
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/

 

域名访问服务请参考 ::  kubernetes + istio进行流量管理

k8s使用中我遇到的问题之后会整理到这里 kubernetes 常见问题整理

转载于:https://www.cnblogs.com/qingfeng2010/p/10540832.html

asdmd78945
关注
k8s install on centos7国内源
tryyourbest0928的博客
12-30 327
https://kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ 需要在每台机器上安装以下的软件包： cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernete
CentOS7利用Kubeadm快速部署Kubernetes集群
郄梦岩的博客
03-30 1539
CentOS7利用Kubeadm快速部署Kubernetes集群
K8S 故障处理经验积累（网络）
热门推荐
会哭的雨@的博客
07-31 1万+
1，cni网络问题导致coredns起不来 NetworkPlugin cni failed to set up pod "coredns-5c98db65d4-fr9nk_kube-system" network: failed to set bridge addr: "cni0" already has an IP address different from 172.21.0.1/24NetworkPlugin cni failed to set up pod "coredns-5c98db65d
【k8s集群故障】unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf
最新发布
一起加油吧
09-11 652
如果故障情况恢复正常，则kubelet或节点控制器可以删除相关的污点。具体文档地址，如下所示：https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/发现 unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf。将master节点的/etc/kubernetes/admin.conf拷贝到node-1。
阿里云搭建k8s kubeadm init失败的原因
cxfTrue
09-17 7446
使用阿里云搭建k8s时，在主节点执行kubeadm init时候卡在 Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0 这是因为kubeadm init 指定了"--apiserver-advertise-address"为公网ip，但是阿里云的机器是vpc网络，使用ifconfig
k8s - kubelet启动失败处理记录
longway111的博客
08-04 2465
看看日志吧：journalctl -xu kubelet。google了下，应该是证书过期了。
【云原生|Kubernetes】02-Kubeadm安装部署Kubernetes集群
qq_43714097的博客
05-04 2118
前面简单介绍了Kubernetes的相关架构及其组件，这一章节我们将介绍如何快速搭建Kubernetes的集群，这里我们将使用官方支持的用于部署 Kubernetes 的工具—kubeadm。主机名ip地址节点类型系统版本centos7.xnode1centos7.xnode2centos7.x在所有节点上安装kubeadm，kubectl，kubelet。kubeadm： 安装kubernetes集群的安装工具，（安装master和node上的组件）
CentOS7使用Kubeadm快速搭建Kubernetes集群
"本文档是关于在CentOS7系统中使用Kubeadm工具安装和部署Kubernetes（K8S）集群的技术方案，由张志龙编写于2017年12月19日。" 在CentOS7环境下，Kubernetes的安装通常采用kubeadm工具，它为用户提供了方便快捷的...
Centos7 使用kubeadm部署kubernetes集群用到的那些yum源
运维拯救世界
10-11 180
公网环境下Centos7 使用kubeadm部署kubernetes集群用到的那些yum源。
CentOS 7下使用Kubeadm部署最新版Kubernetes
03-27 1347
个人把玩用的，不推荐生产环境使用Kubeadm部署哈，生产环境可以使用国人开源的kubeasz 前言 安装这个花了我两天的时间，想想过程就非常的难受啊，各种小问题层出不穷，最开始打算在阿里云上面搭，但是我阿里云的三台机器是用我家人的手机号买的（双十一活动，哈哈），可这也就导致了搭建失败，网络不通，k8s只会绑定内网IP，无法绑定外网IP，而阿里云不同账号的内网是不互通的，可以搞但是得花钱，一说花钱，那。。我必不可能花钱，于是果断在本地搭了三台虚拟机，在虚拟机上搭建的过程虽然没有网络的问题了，但零零碎碎.
CentOS7下快速部署kubernetes1.15高可用集群-kubeadm篇.pdf
01-03
kubernetes官方给我提供了一个简易的集群部署工具Kubeadm，目的就是 让我们能快速的部署一个可用的生产环境集群。今天我就和大家一起来体验一 下
保姆级Ubuntu20/22安装Kubernetes与踩坑
qq_40851623的博客
11-30 2201
Kubernetes：1.25.10 （1.24+已经弃用了docker，初始化时会有超时异常）
k8s-常见问题汇总
因上努力，果上随缘。但行好事，莫问前程。
04-27 4526
问题1：单node上容器网段pod被隔离 现象： 1)生产环境节点node_A重启后，上面有几个pod_1(pod namespace的pod) 的init container初始化失败，报错连接svc/kubernetes(apiserver的service) timed out 2)该节点上：容器网络ns(namespace)的pod访问不了自己的GW(GW1)，node网络ns的可以访问自己的GW(GW2) 3)容器网络ns的pod可以ping通自己node_A，但是ping不通其他node 4)物
Kubernetes_核心组件_kubelet_kubelet服务全解析
毛毛的专栏
05-28 3389
kubelet所有知识点查看kubelet当前运行systemctl status kubelet -l 搞懂所有属性 (journalctl -u kubelet.service)kubelet配置文件 cat /var/lib/kubelet/config.yamlkubelet启动参数文件 10-kubeadm.confkubelet启动全过程 (自定义启动参数文件)kubelet和pause镜像的关系就是Pod内容器间通信。
[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz‘ failed with error:
ck784101777的博客
11-20 1万+
可能性1:10-kubeadm.conf文件缺失 vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf # Note: This dropin only works with kubeadm and kubelet v1.11+ [Service] Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.con
三、公网环境搭建Kubernetes (k8s) 集群的详细图解
胖太乙
06-04 4938
上一节介绍了 Kubernetes (k8s) 内网集群的搭建详细图解 ，也介绍了云服务器按量付费的租赁方式，但是这种方式有点不好之处就是，每次停机重启之后 ， IP地址就变了，导致XShell 连接工具每次要改IP， 很是麻烦， 好在我这个人比较能薅羊毛，于是在腾讯云， 百度云， 阿里云上面都薅了一波羊毛。【想薅羊毛看这里：“羊毛”】。所以趁着休息，将手头的几台云服务器搭建成 k8s 集群，由于这几台云服务属于不同的云服务厂商，无法搭建局域网环境的 k8s 集群，故笔者搭建的是公网环境的 k8s 集群。我
centos7 使用 kubeadm 安装 kubernetes
Man_In_The_Night的博客
11-19 605
centos7 使用 kubeadm 安装 kubernetes 时间: 20191118 系统: CentOS Linux release 7.6.1810 vm: k8s-master: 192.168.116.4 k8s-node01:192.168.116.5 使用最新版的 kubeadm [root@k8s-master ~]# kubeadm config images list W1...
使用kubeadm快速部署Kubernetes集群指南
"kubeadm部署Kubernetes集群" 在部署Kubernetes集群时，kubeadm是一个非常实用的工具，尤其对于新手来说，它提供了一种简单易用的方法来快速搭建集群。kubeadm是Kubernetes项目的一部分，旨在简化集群的初始化和...
K8S 容器绑核
m0_67015366的博客
05-15 1868
最近想学习在K8S环境给容器绑定CPU，曾经参考过一篇帖子，绑核成功，但第二次操作就莫名其妙再没每成功过。 又找了几篇帖子，而且还把老的v1.17.4卸了，重装一个v1.20.15，再次试验。 原来的操作 1，停止kubelet（未进行节点驱逐的操作） 2，修改config.yaml （/var/lib/kubelet/config.yaml）添加 cpuManagerPolicy: static 3，删除cpu_manager_state 4，systemctl restart kubelet
K8s集群搭建及相关问题
qq_42506719的博客
04-15 988
任意节点访问 http://192.168.75.128:80 均可，用户名：admin，密码：Kuboard123。(2) 填入集群名称、IP和token等信息，将集群导入Kuboard;（注意：此时先不要将slave节点加入集群，因为还未配置flannel实现各节点网络通信！查看原因（注意：此处具体问题有很多，大家可以根据实际情况寻找不同的解决方案），发现以下问题。，不然加入集群时可能会出现后面问题3——flannel导致的node节点。7.2 查看dashboard创建的pods运行情况；
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值