k8s--kubeadm 搭建k8s

最新推荐文章于 2024-03-07 09:36:36 发布

qq_26496877

最新推荐文章于 2024-03-07 09:36:36 发布

阅读量423

点赞数

分类专栏： docker&k8s&devops

原文链接：https://www.cnblogs.com/double-dong/p/11483670.html

版权

docker&k8s&devops 专栏收录该内容

6 篇文章 0 订阅

订阅专栏

一、Kubernetes概述

1.1 Kubernetes是什么

Kubernetes是Google在2014年开源的一个容器集群管理系统，Kubernetes简称K8S。

K8S用于容器化应用程序的部署，扩展和管理。

K8S提供了容器编排，资源调度，弹性伸缩，部署管理，服务发现等一系列功能。

Kubernetes目标是让部署容器化应用简单高效。

1.2 Kubernetes特性

自我修复

在节点故障时重新启动失败的容器，替换和重新部署，保证预期的副本数量；杀死健康检查失败的容器，并且在未准备好之前不会处理客户端请求，确保线上服务不中断。

弹性伸缩

使用命令、UI或者基于CPU使用情况自动快速扩容和缩容应用程序实例，保证应用业务高峰并发时的高可用性；业务低峰时回收资源，以最小成本运行服务。

自动部署和回滚

K8S采用滚动更新策略更新应用，一次更新一个Pod，而不是同时删除所有Pod，如果更新过程中出现问题，将回滚更改，确保升级不受影响业务。

服务发现和负载均衡

K8S为多个容器提供一个统一访问入口（内部IP地址和一个DNS名称），并且负载均衡关联的所有容器，使得用户无需考虑容器IP问题。

机密和配置管理

管理机密数据和应用程序配置，而不需要把敏感数据暴露在镜像里，提高敏感数据安全性。并可以将一些常用的配置存储在K8S中，方便应用程序使用。

存储编排

挂载外部存储系统，无论是来自本地存储，公有云（如AWS），还是网络存储（如NFS、GlusterFS、Ceph）都作为集群资源的一部分使用，极大提高存储使用灵活性。

批处理

提供一次性任务，定时任务；满足批量数据处理和分析的场景。

1.3 Kubernetes集群架构与组件

在这里插入图片描述

1.4 Kubernetes集群组件介绍

1.4.1 Master组件

kube-apiserver

Kubernetes API，

集群的统一入口
，各组件协调者，以RESTful API提供接口服务，所有对象资源的增删改查和监听操作都交给APIServer处理后再提交给Etcd存储。

kube-controller-manager

处理集群中常规后台任务，一个资源对应一个控制器，而ControllerManager就是负责管理这些控制器的。

kube-scheduler

根据调度算法为新创建的Pod选择一个Node节点，可以任意部署,可以部署在同一个节点上,也可以部署在不同的节点上。

etcd

分布式键值存储系统。用于保存集群状态数据，比如Pod、Service等对象信息。

1.4.2 Node组件
kubelet

kubelet是Master在Node节点上的Agent，管理本机运行容器的生命周期，比如创建容器、Pod挂载数据卷、下载secret、获取容器和节点状态等工作。kubelet将每个Pod转换成一组容器。

kube-proxy

在Node节点上实现Pod网络代理，维护网络规则和四层负载均衡工作。

docker或rocket

容器引擎，运行容器。

1.5 Kubernetes 核心概念

在这里插入图片描述

Pod

最小部署单元

一组容器的集合

一个Pod中的容器共享网络命名空间

Pod是短暂的

Controllers

ReplicaSet ：确保预期的Pod副本数量
Deployment ：无状态应用部署
StatefulSet ：有状态应用部署
DaemonSet ：确保所有Node运行同一个Pod
Job ：一次性任务
Cronjob ：定时任务

更高级层次对象，部署和管理Pod

Service
防止Pod失联
定义一组Pod的访问策略
Label ：标签，附加到某个资源上，用于关联对象、查询和筛选
Namespaces：命名空间，将对象逻辑上隔离
Annotations ：注释

二、kubeadm 快速部署K8S集群

2.1 kubernetes 官方提供的三种部署方式

minikube

Minikube是一个工具，可以在本地快速运行一个单点的Kubernetes，仅用于尝试Kubernetes或日常开发的用户使用。部署地址：https://kubernetes.io/docs/setup/minikube/

kubeadm

Kubeadm也是一个工具，提供kubeadm init和kubeadm join，用于快速部署Kubernetes集群。部署地址：https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/

二进制包

推荐，从官方下载发行版的二进制包，手动部署每个组件，组成Kubernetes集群。下载地址：https://github.com/kubernetes/kubernetes/releases

2.2 安装kubeadm环境准备

以下操作,在三台节点都执行

2.2.1 环境需求

环境：centos 7.4 +

硬件需求：CPU>=2c ,内存>=2G

2.2.2 环境角色

IP 角色安装软件

2.2.3 环境初始化

PS : 以下所有操作，在三台节点全部执行

1、关闭防火墙及selinux

systemctl stop firewalld && systemctl disable firewalld
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config  && setenforce 0

2、关闭 swap 分区

swapoff -a # 临时

 sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab #永久

设置/etc/hosts

 cat >> /etc/hosts << EOF
 192.168.48.61 master
 192.168.48.62 node01
 192.168.48.63 node02
 EOF

内核调整,将桥接的IPv4流量传递到iptables的链

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

设置系统时区并同步时间服务器

yum install -y ntpdate
ntpdate time.windows.com

2.2.4 docker 安装

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version
Docker version 18.06.1-ce, build e68fc7a

docker 初始化

 cat > /etc/docker/daemon.json << EOF
 {
   "data-root":"/data/docker",
   "insecure-registries":["registry.access.redhat.com","quay.io"], 
   "exec-opts":["native.cgroupdriver=systemd"], 
   "live-restore":true,
   "bip":"172.0.100.1/24",
  "registry-mirrors":["https://q2gr04ke.mirror.aliyuncs.com"] 
}
EOF

“exec-opts”:[“native.cgroupdriver=systemd”] 这个配置项对于docker很重要

2.2.5 添加kubernetes YUM软件源

 cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

2.2.6 安装kubeadm,kubelet和kubectl

 yum install -y kubelet-1.15.0 kubeadm-1.15.0 kubectl-1.15.0
 systemctl enable kubelet

2.3 部署Kubernetes Master

 kubeadm init \
--apiserver-advertise-address=192.168.48.61 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.15.0 \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16

如果对init的结果不满意，可使用kubeadm reset 进行重置，并且停掉相关的进程，然后再次使用上面的命令。

[init] Using Kubernetes version: v1.15.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.1.0.1 192.168.48.61]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [master localhost] and IPs [192.168.48.61 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [master localhost] and IPs [192.168.48.61 127.0.0.1 ::1]
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 20.003894 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: iv8whr.mz4mwki34oeyb985
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.48.61:6443 --token iv8whr.mz4mwki34oeyb985 \
    --discovery-token-ca-cert-hash sha256:7e54dfe36303dc5beb3bbcaf2aafe0edb24b8f84f7632cb647c9c0817edbd2a9

根据提示进行如下操作

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

2.4节点加入集群

默认token的有效期为24小时，当过期之后，该token就不可用了，如果后续有nodes节点加入，解决方法如下：
重新生成新的token

kubeadm token create

kubeadm token list

获取ca证书sha256编码hash值

 openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //

节点加入集群

kubeadm join --token aa78f6.8b4cafc8ed26c34f --discovery-token-ca-cert-hash sha256:0fd95a9bc67a7bf0ef42da968a0d55d92e52898ec37c971bd77ee501d845b538 192.168.73.138:6443 --skip-preflight-chec

kubeadm join --token 9xdks8.0mn3jot1pzk4c6iw --discovery-token-ca-cert-hash sha256:7e54dfe36303dc5beb3bbcaf2aafe0edb24b8f84f7632cb647c9c0817edbd2a9 192.168.48.61:6443 --skip-preflight-chec

2.5 安装flannel

只需要在Master 节点执行

wget https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml

kubectl apply -f kube-flannel.yml

2.6 测试k8s 集群

[root@k8s-master ~]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created

[root@k8s-master ~]# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed

[root@k8s-master ~]# kubectl get pods,svc
NAME                         READY   STATUS    RESTARTS   AGE
pod/nginx-554b9c67f9-wf5lm   1/1     Running   0          24s

NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.1.0.1       <none>        443/TCP        39m
service/nginx        NodePort    10.1.224.251   <none>        80:31745/TCP   9

2.7 安装dashboard

[root@k8s-master ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

[root@k8s-master ~]# vim kubernetes-dashboard.yaml
修改内容：
109     spec:
110       containers:
111       - name: kubernetes-dashboard
112         image: lizhenliang/kubernetes-dashboard-amd64:v1.10.1   # 修改此行

......

157 spec:
158   type: NodePort     # 增加此行
159   ports:
160     - port: 443
161       targetPort: 8443
162       nodePort: 30001   # 增加此行
163   selector:
164     k8s-app: kubernetes-dashboard

[root@k8s-master ~]# kubectl apply -f kubernetes-dashboard.yaml

创建service account并绑定默认cluster-admin管理员集群角色：

[root@k8s-master ~]# kubectl create serviceaccount dashboard-admin -n kube-system
serviceaccount/dashboard-admin created
[root@k8s-master ~]# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin
--serviceaccount=kube-system:dashboard-admin
[root@k8s-master ~]# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')
Name:         dashboard-admin-token-d9jh2
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: dashboard-admin
              kubernetes.io/service-account.uid: 4aa1906e-17aa-4880-b848-8b3959483323

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  11 bytes
token:      eyJhbGciOiJ...(省略如下)...AJdQ

token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tZDlqaDIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNGFhMTkwNmUtMTdhYS00ODgwLWI4NDgtOGIzOTU5NDgzMzIzIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.OkF6h7tVQqmNJniCHJhY02G6u6dRg0V8PTiF8xvMuJJUphLyWlWctgmplM4kjKVZo0fZkAthL7WAV5p_AwAuj4LMfo1X5IpxUomp4YZyhqgsBM0A2ksWoKoLDjbizFwOty8TylWlsX1xcJXZjmP9OvNgjjSq5J90N5PnxYIIgwAMP3fawTP7kUXxz5WhJo-ogCijJCFyYBHoqHrgAbk9pusI8DpGTNIZxBMxkwPPwFwzNCOfKhD0c8HjhNeliKsOYLryZObRdmTQXmxsDfxynTKsRxv_EPQb99yW9GXJPQL0OwpYb4b164CFv857ENitvvKEOU6y55P9hFkuQuAJdQ

当匿名账户无法查看看k8s集群中的某些资源时，需要执行以下命令：

 kubectl create clusterrolebinding test:anonymous --clusterrole=cluster-admin --user=system:anonymous

在火狐浏览器访问(google受信任问题不能访问)地址： https://NodeIP:30001
在这里插入图片描述

查看dashboard 时需要的token

 kubectl -n kube-system describe $(kubectl -n kube-system get secret -n kube-system -o name | grep namespace) | grep token

解决只能火狐访问的问题

[root@k8s-master ~]# cd /etc/kubernetes/pki/
[root@k8s-master pki]# mkdir ui
[root@k8s-master pki]# cp apiserver.crt  ui/
[root@k8s-master pki]# cp apiserver.key  ui/
[root@k8s-master pki]# cd ui/
[root@k8s-master ui]# mv apiserver.crt dashboard.pem
[root@k8s-master ui]# mv  apiserver.key   dashboard-key.pem
[root@k8s-master ui]# kubectl delete secret kubernetes-dashboard-certs -n kube-system
[root@k8s-master ui]# kubectl create secret generic kubernetes-dashboard-certs --from-file=./ -n kube-system

[root@k8s-master]# vim kubernetes-dashboard.yaml #回到这个yaml的路径下修改
修改 dashboard-controller.yaml 文件，在args下面增加证书两行
          - --tls-key-file=dashboard-key.pem
          - --tls-cert-file=dashboard.pem
[root@k8s-master ~]kubectl apply -f kubernetes-dashboard.yaml
[root@k8s-master ~]# kubectl create serviceaccount dashboard-admin -n kube-system
serviceaccount/dashboard-admin created
[root@k8s-master ~]# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin
--serviceaccount=kube-system:dashboard-admin
[root@k8s-master ~]# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')
Name:         dashboard-admin-token-zbn9f
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: dashboard-admin
              kubernetes.io/service-account.uid: 40259d83-3b4f-4acc-a4fb-43018de7fc19

Type:  kubernetes.io/service-account-token

Data
====
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4temJuOWYiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNDAyNTlkODMtM2I0Zi00YWNjLWE0ZmItNDMwMThkZTdmYzE5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.E0hGAkeQxd6K-YpPgJmNTv7Sn_P_nzhgCnYXGc9AeXd9k9qAcO97vBeOV-pH518YbjrOAx_D6CKIyP07aCi_3NoPlbbyHtcpRKFl-lWDPdg8wpcIefcpbtS6uCOrpaJdCJjWFcAEHdvcfmiFpdVVT7tUZ2-eHpRTUQ5MDPF-c2IOa9_FC9V3bf6XW6MSCZ_7-fOF4MnfYRa8ucltEIhIhCAeDyxlopSaA5oEbopjaNiVeJUGrKBll8Edatc7-wauUIJXAN-dZRD0xTULPNJ1BsBthGQLyFe8OpL5n_oiHM40tISJYU_uQRlMP83SfkOpbiOpzuDT59BBJB57OQtl3w
ca.crt:     1025 bytes

qq_26496877

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
k8s--kubeadm 搭建k8s

一、Kubernetes概述1.1 Kubernetes是什么Kubernetes是Google在2014年开源的一个容器集群管理系统，Kubernetes简称K8S。K8S用于容器化应用程序的部署，扩展和管理。K8S提供了容器编排，资源调度，弹性伸缩，部署管理，服务发现等一系列功能。Kubernetes目标是让部署容器化应用简单高效。1.2 Kubernetes特性自我修复在节点故障时重新启动失败的容器，替换和重新部署，保证预期的副本数量；杀死健康检查失败的容器，并且在未准备好之前不会处理
复制链接

扫一扫