一、kubernetes
kubernetes是为容器服务而生的一个可移植容器的编排管理工具,简单的说就是用户通过master控制node。ubernete8位字符被8代替简称k8s.
k8s主要由两部分构成:
- Master
k8s集群控制节点,对集群进行调度管理,接受集群外用户去集群操作请求;
Master Node 由 API Server、Scheduler、ClusterState Store(ETCD 数据库)和 Controller MangerServer 所组成;
- Nodes
集群工作节点,运行用户业务应用容器;
Nodes节点也叫Worker Node,包含kubelet、kube proxy 和 Pod(Container Runtime);
- 官网:https://www.kubernetes.io
- 源码:github:https://github.com/kubernetes/kubernetes
二、安装环境
k8s官方介绍了多种安装方式
- minikube:本地安装minikube单点集群,本地使用,不适用于服务器:官方有测试工具,网址:https://kubernetes.io/docs/tutorials/hello-minikube/
- kind :类似minikube,网址:https://kind.sigs.k8s.io/
- kubeadm:Kubeadm是一个K8s部署工具,提供kubeadm init 和 kubeadm join两个操作命令,可以快速部署一个Kubernetes集群.推荐使用
- 二进制包:在Github下载二进制包安装,流程比较麻烦
- yum安装:通过yum安装Kubernetes的每个组件,组成Kubernetes集群,类似二进制安装,但源已比较老。
- 第三方工具.
- 直接购买
三、kubeadm的方式进行安装
准备:三台测试用服务器,条件不具备的情况下,两台也可。最低配置需CPU2核,2G以上内存。我准备当前环境为CentOS7.5,官方推荐CentOS7以上版本.
- 环境准备
# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
# 关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config #永久
setenforce 0 #临时
# 关闭swap(k8s禁止虚拟内存以提高性能)
sed -ri 's/.*swap.*/#&/' /etc/fstab #永久
swapoff -a #临时
# 在master添加hosts,其中11为master的内网ip,15和3为node的内网ip
cat >> /etc/hosts << EOF
172.21.0.11 k8smaster
172.21.0.15 k8snode1
172.21.0.3 k8snode2
EOF
或者:
vi /etc/hosts
172.21.0.11 k8smaster
172.21.0.15 k8snode1
172.21.0.3 k8snode2
# 设置网桥参数
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system #生效
# 时间同步
yum install ntpdate -y #linux下使服务器时间同步
ntpdate time.windows.com
安装 Docker :yum install docker -y (这个安装的Docker版本偏旧) 1.13.x
更新docker的yum源后安装19.03.13版,本系列都用19.03.13版
yum install wget -y
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
#安装指定版本的docker:
yum install docker-ce-19.03.13 -y
#安装成功后提示如下
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
docker-ce-stable | 3.5 kB 00:00:00
(1/2): docker-ce-stable/7/x86_64/primary_db | 58 kB 00:00:01
(2/2): docker-ce-stable/7/x86_64/updateinfo | 55 B 00:00:02
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.13-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.13-3.el7.x86_64
--> Processing Dependency: containerd.io >= 1.2.2-3 for package: 3:docker-ce-19.03.13-3.el7.x86_64
--> Processing Dependency: docker-ce-cli for package: 3:docker-ce-19.03.13-3.el7.x86_64
--> Processing Dependency: libcgroup for package: 3:docker-ce-19.03.13-3.el7.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.119.2-1.911c772.el7_8 will be installed
--> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.119.2-1.911c772.el7_8.noarch
---> Package containerd.io.x86_64 0:1.4.4-3.1.el7 will be installed
---> Package docker-ce-cli.x86_64 1:20.10.5-3.el7 will be installed
---> Package libcgroup.x86_64 0:0.41-21.el7 will be installed
--> Running transaction check
---> Package policycoreutils-python.x86_64 0:2.5-34.el7 will be installed
--> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-34.el7.x86_64
--> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-34.el7.x86_64
--> Running transaction check
---> Package audit-libs-python.x86_64 0:2.8.5-4.el7 will be installed
---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed
---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed
---> Package python-IPy.noarch 0:0.75-6.el7 will be installed
---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===============================================================================================================================================================================================
Package Arch Version Repository Size
===============================================================================================================================================================================================
Installing:
docker-ce x86_64 3:19.03.13-3.el7 docker-ce-stable 24 M
Installing for dependencies:
audit-libs-python x86_64 2.8.5-4.el7 os 76 k
checkpolicy x86_64 2.5-8.el7 os 295 k
container-selinux noarch 2:2.119.2-1.911c772.el7_8 extras 40 k
containerd.io x86_64 1.4.4-3.1.el7 docker-ce-stable 33 M
docker-ce-cli x86_64 1:20.10.5-3.el7 docker-ce-stable 33 M
libcgroup x86_64 0.41-21.el7 os 66 k
libsemanage-python x86_64 2.5-14.el7 os 113 k
policycoreutils-python x86_64 2.5-34.el7 os 457 k
python-IPy noarch 0.75-6.el7 os 32 k
setools-libs x86_64 3.3.8-4.el7 os 620 k
Transaction Summary
===============================================================================================================================================================================================
Install 1 Package (+10 Dependent packages)
Total download size: 92 M
Installed size: 391 M
Downloading packages:
(1/11): container-selinux-2.119.2-1.911c772.el7_8.noarch.rpm | 40 kB 00:00:00
(2/11): audit-libs-python-2.8.5-4.el7.x86_64.rpm | 76 kB 00:00:00
(3/11): checkpolicy-2.5-8.el7.x86_64.rpm | 295 kB 00:00:00
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/docker-ce-19.03.13-3.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY ] 1.7 MB/s | 31 MB 00:00:35 ETA
Public key for docker-ce-19.03.13-3.el7.x86_64.rpm is not installed
(4/11): docker-ce-19.03.13-3.el7.x86_64.rpm | 24 MB 00:00:15
(5/11): libcgroup-0.41-21.el7.x86_64.rpm | 66 kB 00:00:00
(6/11): libsemanage-python-2.5-14.el7.x86_64.rpm | 113 kB 00:00:00
(7/11): python-IPy-0.75-6.el7.noarch.rpm | 32 kB 00:00:00
(8/11): policycoreutils-python-2.5-34.el7.x86_64.rpm | 457 kB 00:00:00
(9/11): setools-libs-3.3.8-4.el7.x86_64.rpm | 620 kB 00:00:00
(10/11): docker-ce-cli-20.10.5-3.el7.x86_64.rpm | 33 MB 00:01:09
(11/11): containerd.io-1.4.4-3.1.el7.x86_64.rpm | 33 MB 00:01:34
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 1.0 MB/s | 92 MB 00:01:34
Retrieving key from https://download.docker.com/linux/centos/gpg
Importing GPG key 0x621E9F35:
Userid : "Docker Release (CE rpm) <docker@docker.com>"
Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
From : https://download.docker.com/linux/centos/gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libcgroup-0.41-21.el7.x86_64 1/11
Installing : setools-libs-3.3.8-4.el7.x86_64 2/11
Installing : audit-libs-python-2.8.5-4.el7.x86_64 3/11
Installing : checkpolicy-2.5-8.el7.x86_64 4/11
Installing : python-IPy-0.75-6.el7.noarch 5/11
Installing : libsemanage-python-2.5-14.el7.x86_64 6/11
Installing : policycoreutils-python-2.5-34.el7.x86_64 7/11
Installing : 2:container-selinux-2.119.2-1.911c772.el7_8.noarch 8/11
setsebool: SELinux is disabled.
Installing : containerd.io-1.4.4-3.1.el7.x86_64 9/11
Installing : 1:docker-ce-cli-20.10.5-3.el7.x86_64 10/11
Installing : 3:docker-ce-19.03.13-3.el7.x86_64 11/11
Verifying : 1:docker-ce-cli-20.10.5-3.el7.x86_64 1/11
Verifying : libsemanage-python-2.5-14.el7.x86_64 2/11
Verifying : 3:docker-ce-19.03.13-3.el7.x86_64 3/11
Verifying : 2:container-selinux-2.119.2-1.911c772.el7_8.noarch 4/11
Verifying : python-IPy-0.75-6.el7.noarch 5/11
Verifying : checkpolicy-2.5-8.el7.x86_64 6/11
Verifying : policycoreutils-python-2.5-34.el7.x86_64 7/11
Verifying : containerd.io-1.4.4-3.1.el7.x86_64 8/11
Verifying : audit-libs-python-2.8.5-4.el7.x86_64 9/11
Verifying : setools-libs-3.3.8-4.el7.x86_64 10/11
Verifying : libcgroup-0.41-21.el7.x86_64 11/11
Installed:
docker-ce.x86_64 3:19.03.13-3.el7
Dependency Installed:
audit-libs-python.x86_64 0:2.8.5-4.el7 checkpolicy.x86_64 0:2.5-8.el7 container-selinux.noarch 2:2.119.2-1.911c772.el7_8 containerd.io.x86_64 0:1.4.4-3.1.el7
docker-ce-cli.x86_64 1:20.10.5-3.el7 libcgroup.x86_64 0:0.41-21.el7 libsemanage-python.x86_64 0:2.5-14.el7 policycoreutils-python.x86_64 0:2.5-34.el7
python-IPy.noarch 0:0.75-6.el7 setools-libs.x86_64 0:3.3.8-4.el7
Complete!
#然后启动docker
systemctl start docker
安装kubeadm,kubelet,kubectl
yum install kubelet-1.19.4 -y
yum install kubeadm-1.19.4 -y
yum install kubectl-1.19.4 -y
注:自动开机启动docker,k8s
systemctl enable docker.service
systemctl enable kubelet.service
以上安装需在三台服务器执行,重启动三台服务器-使配置生效
然后部署Kubernetes Master主节点
kubeadm init --apiserver-advertise-address=172.21.0.11 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.19.4 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16
命令执行后,最下方显示:
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 172.21.0.11:6443 --token trxa6y.8modhd2t73ivmhej \
--discovery-token-ca-cert-hash sha256:febf9073febf9073febf9073febf9073febf9073
即为安装成功。
kubeadm join 172.21.0.11:6443 --token trxa6y.8modhd2t73ivmhej \ --discovery-token-ca-cert-hash sha256:febf9073febf9073febf9073febf9073febf9073 为node 加入master的执行命令和加密数据,需要记录。
说明:
service-cidr 的选取不能和PodCIDR及本机网络有重叠或者冲突,一般可以选择一个本机网络和PodCIDR都没有用到的私网地址段,比如PODCIDR使用10.244.0.0/16, 那么service cidr可以选择10.96.0.0/12,网络无重叠冲突即可;
接下来在master机器上执行:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@VM-0-11-centos home]# mkdir -p $HOME/.kube
. .. .bash_history .bash_logout .bash_profile .bashrc .cache .config .cshrc .kube .pip .pki .pydistutils.cfg .ssh .tcshrc .viminfo
[root@VM-0-11-centos ~]# echo $HOME
/root
[root@VM-0-11-centos ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@VM-0-11-centos ~]# chown $(id -u):$(id -g) $HOME/.kube/config
[root@VM-0-11-centos ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
vm-0-11-centos NotReady master 11m v1.19.4
Node端加入集群,执行master端刚才的提示命令即可:
kubeadm join 172.21.0.11:6443 --token trxa6y.8modhd2t73ivmhej \
--discovery-token-ca-cert-hash sha256:febf9073febf9073febf9073febf9073febf9073
提示:
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
安装成功:根据提示执行kubectl get nodes查看已安装成功的node
查看node状态
[root@VM-0-11-centos ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
vm-0-11-centos NotReady master 14m v1.19.4
vm-0-15-centos NotReady <none> 2m8s v1.19.4
vm-0-3-centos NotReady <none> 2m1s v1.19.4
下载kube-flannel.yml文件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
应用kube-flannel.yml文件得到运行时容器
kubectl apply -f kube-flannel.yml #在master机器上执行
[root@VM-0-11-centos ~]# kubectl apply -f kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
应用成功
然后查看节点状态: kubectl get nodes (在master机器上执行)
NAME STATUS ROLES AGE VERSION
vm-0-11-centos Ready master 22m v1.19.4
vm-0-15-centos Ready <none> 10m v1.19.4
vm-0-3-centos Ready <none> 10m v1.19.4
至此k8s搭建成功