集群环境准备
操作系统 | IP | 角色 | CPU核心数 | 运行内存 | Hostname |
---|---|---|---|---|---|
CentOS 7.8 | 192.168.1.2 | Master | 2 | 2G | k8s-master |
CentOS 7.8 | 192.168.1.3 | Nade1 | 4 | 8G | k8s-nade1 |
CentOS 7.8 | 192.168.1.4 | Nade2 | 4 | 8G | k8s-nade2 |
CentOS 7.8 | 192.168.1.5 | Nade3 | 4 | 8G | k8s-nade3 |
CentOS 7.8 | 192.168.1.6 | Extend | 8 | 16G | k8s-extend |
基本环境安装
时区设置
包含节点:全部
所有节点调整时区(因为时区不正确的话,会影响令牌有效性校验)
yum -y install ntp
#通过阿里云时间服务器校准时间
ntpdate ntp1.aliyun.com
# 设置时区为上海
timedatectl set-timezone Asia/Shanghai
设置后,使用timedatectl命令查看下时间是否和当前北京时间相符合
安装Docker
包含节点:全部
安装必要依赖:
yum install -y yum-utils \
device-mapper-persistent-data \
lvm2
添加docker稳定版仓库:
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
安装18.09版本:
yum -y install docker-ce-18.09.0 docker-ce-cli-18.09.0 containerd.io
安装19.03.5版本:
yum -y install docker-ce-19.03.5 docker-ce-cli-19.03.5 containerd.io
启动Docker,并设置开机自启:
systemctl enable docker && systemctl start docker
如果Docker是纯净环境请执行:
cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": [
"https://dockerhub.azk8s.cn",
"https://reg-mirror.qiniu.com",
"https://registry.docker-cn.com",
"https://c3tt2fj9.mirror.aliyuncs.com"
]
}
EOF
否则执行 vi/etc/docker/daemon.json文件 手动添加如下:
{
"registry-mirrors": [
"https://dockerhub.azk8s.cn",
"https://reg-mirror.qiniu.com",
"https://registry.docker-cn.com",
"https://c3tt2fj9.mirror.aliyuncs.com"
]
}
重启Docker
systemctl daemon-reload
systemctl restart docker
验证是否安装成功:
[root@k8s-master ~]# docker -v
Docker version 18.09.0, build 4d60db4
K8S环境安装
安装准备
包含节点:Master、Node1、Node2、Node3
修改节点主机名称:
hostnamectl set-hostname XXX
安装必要软件:
yum install -y net-tools.x86_64 wget
配置hosts:
cat >> /etc/hosts << EOF
192.168.1.2 k8s-master
192.168.1.3 k8s-node1
192.168.1.4 k8s-node2
192.168.1.5 k8s-node3
EOF
安装 nfs-utils:
必须先安装 nfs-utils 才能挂载 nfs 网络存储
yum install -y nfs-utils
关闭防火墙:
为了避免kubernetes的Master节点和各个工作节点的Node节点间的通信出现问题,我们可以关闭本地搭建的Centos虚拟机的防火墙。
systemctl disable firewalld
systemctl stop firewalld
禁用SELinux,让容器可以顺利地读取主机文件系统:
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
关闭swap:
Swap是操作系统在内存吃紧的情况申请的虚拟内存,按照Kubernetes官网的说法,Swap会对Kubernetes的性能造成影响,不推荐使用Swap。
echo "vm.swappiness = 0">> /etc/sysctl.conf
swapoff -a
修改Docker配置:
vi /etc/docker/daemon.json
在{}内追加如下内容:
"exec-opts": ["native.cgroupdriver=systemd"]
重启Docker:
systemctl daemon-reload
systemctl restart docker
将桥接的IPv4流量传递到iptables的链:
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
安装kubeadm等工具
包含节点:Master、Node1、Node2、Node3
配置国内的kubernetes源:
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
安装kubelet、kubeadm和kubectl工具:
yum install -y kubelet-1.22.0 kubeadm-1.22.0 kubectl-1.22.0 --disableexcludes=kubernetes
--disableexcludes=kubernetes 禁掉除了这个之外的别的仓库
启动kubelet并设置开机自启:
systemctl enable kubelet && systemctl start kubelet
安装Master
包含节点:Master
初始化Master:
kubeadm init --kubernetes-version=v1.22.0 \
--pod-network-cidr=10.244.0.0/16 \
--service-cidr=10.1.0.0/16 \
--apiserver-advertise-address=192.168.1.2 \
--image-repository registry.aliyuncs.com/google_containers
开放公网IP初始化Master:
kubeadm init --kubernetes-version=v1.22.0 \
--pod-network-cidr=10.244.0.0/16 \
--service-cidr=10.1.0.0/16 \
--apiserver-cert-extra-sans=47.98.168.125 \
--apiserver-advertise-address=192.168.1.2 \
--image-repository registry.aliyuncs.com/google_containers
配置含义如下:
- kubernetes-version: 用于指定k8s版本,这里指定为最新的v1.22.0版本;
- apiserver-advertise-address:用于指定kube-apiserver监听的ip地址,就是master本机IP地址。
- pod-network-cidr:因为后面我们选择flannel作为Pod的网络插件,所以这里需要指定Pod的网络范围为10.244.0.0/16
- service-cidr:用于指定SVC的网络范围;
- image-repository: 其中默认的镜像仓库k8s.gcr.io没有科学上网的话无法访问,我们可以将它修改为国内的阿里镜像仓库registry.aliyuncs.com/google_containers
启动时,需要拉取镜像,过程比较缓慢耐心等待即可。如果你想先拉好镜像再启动,你可以使用kubeadm config images list命令列出需要拉取的镜像
启动成功后,你会看到类似如下提示:
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.1.2:6443 --token ski1in.p36vus9zl5fpr40q \
--discovery-token-ca-cert-hash sha256:148a5745747cbc47a9d3d6dc690fb75c23758e241b862abd5b84c64bb901p61g
意思是,初始化成功,要开始使用K8S集群的话,需要执行以下命令::
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
而下面这段则是用于工作节点Node加入Master集群用的,后面会使用到:
kubeadm join 192.168.1.2:6443 --token ski1in.p36vus9zl5fpr40q \
--discovery-token-ca-cert-hash sha256:148a5745747cbc47a9d3d6dc690fb75c23758e241b862abd5b84c64bb901p61g
##安装Node节点,加入集群包含节点:Node1、Node2、Node3
执行下面这条命令,加入Master:
kubeadm join 192.168.1.2:6443 --token ski1in.p36vus9zl5fpr40q \
--discovery-token-ca-cert-hash sha256:148a5745747cbc47a9d3d6dc690fb75c23758e241b862abd5b84c64bb901p61g
当输出如下内容是说明加入成功:
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
安装网络插件
在master上执行kubectl get nodes命令,会发现Kubernetes提示Master为NotReady状态,这是因为还没有安装网络插件:
[root@iZbp16d6zbtuqktzdobb5vZ ~]kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master NotReady control-plane,master 8m26s v1.22.4
k8s-node1 NotReady <none> 57s v1.22.4
k8s-node2 NotReady <none> 54s v1.22.4
k8s-node3 NotReady <none> 52s v1.22.4
对于网络插件,可以有许多选择,请参考K8S网络插件的说明。这里我选择的flannel
首先下载flannel配置文件:
wget https://kuboard.cn/install-script/flannel/flannel-v0.14.0.yaml
然后根据配置文件安装:
kubectl apply -f ./flannel-v0.14.0.yaml
如下所示,表示安装成功:
[root@k8s-master ~]# kubectl apply -f ./flannel-v0.14.0.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
再次查看节点通信状态:
[root@iZbp16d6zbtuqktzdobb5vZ ~]kubectl get node
k8s-master Ready control-plane,master 9m14s v1.22.4
k8s-node1 Ready <none> 105s v1.22.4
k8s-node2 Ready <none> 102s v1.22.4
k8s-node3 Ready <none> 100s v1.22.4