k8s 集群安装(vagrant + virtualbox + CentOS8)

==============================================

主机环境：windows 11

k8s版本：v1.25

dashboard版本：v2.7.0

calico版本： v3.26.1

CentOS8版本：4.18.0-348.7.1.el8_5.x86_64

用到的脚本：

https://gitcode.net/sundongsdu/k8s_cluster

==============================================

1. Vagrant创建虚拟机节点

首先需要在win11主机上安装vagrant与virtualbox，此处不在赘述。

vagrant默认账号：vagrant/vagrant,  root/vagrant

然后在win11的Vagrantfile文件所在目录下，执行：

vagrant up 创建虚拟机

通过vagrant status查看状态：

2. 通用配置

##################### 所有节点都需要执行的操作 开始 #################
vagrant ssh kubemaster/worker1/worker2
ssh配置：

vi /etc/ssh/sshd_config

PasswordAuthentication yes

然后执行 systemctl restart sshd.service

配置centos yum源：
设置centos8 mirror list:

sudo sed -i -e "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-*

sudo sed -i -e "s|#baseurl=http://mirror.centos.org|baseurl=http://mirrors.aliyun.com|g" /etc/yum.repos.d/CentOS-*

如果需要设置代理：
vi /etc/yum.conf
增加一行：
proxy=http://your-proxy-server:port

### 升级内核：
 
 yum list installed|grep kernel*
 
 yum update kernel -y
 
 yum -y install kernel-devel kernel-headers
 
 yum list installed|grep kernel*
 

删除老的内核，比如：

 rpm -e kernel-4.18.0-240.1.1.el8_3 kernel-core-4.18.0-240.1.1.el8_3 kernel-modules-4.18.0-240.1.1.el8_3 kernel-tools-4.18.0-240.1.1.el8_3 kernel-tools-libs-4.18.0-240.1.1.el8_3
 
 yum list installed|grep kernel*
 
 安装完以后reboot
 
 ==============================================
 此处，安装Guest Additions:
 
 yum -y install libX11 libXt libXext libXmu xorg-x11-server-Xorg
 yum -y install elfutils-libelf-devel gcc make perl
 
 vbox设置->存储里面挂载VBoxGuestAdditions.iso
 

 mkdir -p /mnt/cdrom
 mount /dev/cdrom /mnt/cdrom

 cd /mnt/cdrom
 ./VBoxLinuxAdditions.run
 
 查看：
 [root@kubemaster cdrom]# lsmod|grep vboxguest
vboxguest             409600  2 vboxsf

 
 安装完以后reboot
 
 设置VM共享目录

此时的共享目录为：/media/sf_share_dir/

 !!!  此处执行configure-vm.sh

##################### 所有节点都需要执行的操作 结束 #################
 

 3. master节点配置

##################### master节点需要执行的操作 开始 #################

3.1 集群 init

执行init:
kubeadm init --apiserver-advertise-address=192.168.56.101 --pod-network-cidr=10.244.0.0/16 --image-repository registry.aliyuncs.com/google_containers

执行结果：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.56.101:6443 --token bl52oq.d2qaxxmcw1cl3wnd \
        --discovery-token-ca-cert-hash sha256:75aae5d73711924153d353395365d7bbcd44646f94f7307e58d76b16aebaedc5

如果后续忘记这个join命令，可以在master上重新生成：
kubeadm token create --print-join-command

3.2 安装calico

calico的安装可以参考https://www.cnblogs.com/wangguishe/p/17635391.html

=======================================
安装calico之前可以配置docker镜像加速：

vi /etc/docker/daemon.json
{
  "registry-mirrors": ["https://hub-mirror.c.163.com"]
}

systemctl daemon-reload
systemctl restart docker

如果需要配置代理，可以参考https://blog.csdn.net/weixin_45894220/article/details/130085121
=======================================
calico与k8s的版本对应关系：
https://docs.tigera.io/calico/latest/getting-started/kubernetes/requirements

We test Calico v3.26 against the following Kubernetes versions.

    v1.24
    v1.25
    v1.26
    v1.27

安装calico:   
 kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml
 
 安装成功以后：
 [root@kubemaster vagrant]# kubectl get pod --all-namespaces
NAMESPACE     NAME                                       READY   STATUS    RESTARTS        AGE
kube-system   calico-kube-controllers-74cfc9ffcc-tvkkx   1/1     Running   0               6m7s
kube-system   calico-node-lzb6w                          1/1     Running   0               6m7s
kube-system   coredns-c676cc86f-g89d4                    1/1     Running   0               13h
kube-system   coredns-c676cc86f-n78xv                    1/1     Running   0               13h
kube-system   etcd-kubemaster                            1/1     Running   0               13h
kube-system   kube-apiserver-kubemaster                  1/1     Running   0               13h
kube-system   kube-controller-manager-kubemaster         1/1     Running   1 (3h52m ago)   13h
kube-system   kube-proxy-bw82t                           1/1     Running   0               13h
kube-system   kube-scheduler-kubemaster                  1/1     Running   2 (95s ago)     13h

3.3 安装Metrics-Server

official站点：https://github.com/kubernetes-sigs/metrics-server

wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

修改yaml:

然后执行：

kubectl apply -f components.yaml

此时通过kubectl get pods --all-namespaces可以看到metrics-server已经running

执行kubectl top node可以查看cpu memory:

3.4 安装dashboard

 安装dashboard之前先解除master的限制：
 kubectl taint nodes --all node-role.kubernetes.io/control-plane-
 
 
k8s与dashboard的版本对应：
https://github.com/kubernetes/dashboard/releases

v2.7.0 Compatibility

Kubernetes version	1.22	1.23	1.24	1.25
Compatibility	?	?	?	✓

• ✓ Fully supported version range.
• ? Due to breaking changes between Kubernetes API versions, some features might not work correctly in the Dashboard.

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
修改其中的 imagePullPolicy: IfNotPresent
kubectl apply -f recommended.yaml
 
 kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard
 This will open up a vi screen. Search for the entry ClusterIP and replace it with
NodePort (Case sensitive)

Run the following command to find out the port on which it will listen,
kubectl get svc -n kubernetes-dashboard|grep NodePort|awk '{print
$5}'|cut -f 2 -d :|cut -f 1 -d /

比如此处端口为31768

在vbox上配置端口转发：

创建用户：

参考：https://www.cnblogs.com/wangguishe/p/17582668.html

admin-user.yml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

执行

kubectl apply -f admin-user.yml 

配置secret:

admin-user-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: admin-user-secret
  namespace: kubernetes-dashboard 
  annotations:
    kubernetes.io/service-account.name: admin-user
type: kubernetes.io/service-account-token

执行 

kubectl apply -f admin-user-secret.yaml

查看Secret：

kubectl get secret -n kubernetes-dashboard 

查看token:

kubectl get secret admin-user-secret -n kubernetes-dashboard -o jsonpath={".data.token"} | base64 -d

在windows主机上访问dashboard:

https://127.0.0.1:9443

输入上述的token,

查看工作负载(workloads)注意选择正确的命名空间：

##################### master节点需要执行的操作 结束 #################

4. worker节点配置

##################### worker节点都需要执行的操作 开始 #################
找到上面master节点的join信息并在worker上执行：

kubeadm join 192.168.56.101:6443 --token bl52oq.d2qaxxmcw1cl3wnd \
        --discovery-token-ca-cert-hash sha256:75aae5d73711924153d353395365d7bbcd44646f94f7307e58d76b16aebaedc5

注意，需要等到join之后，systemctl  status kubelet状态才显示为active (running)

##################### worker节点都需要执行的操作 结束 #################

worker节点加入以后，可以在master节点查看nodes:

 查看pods:

如果想查看某个node上的pods，比如worker1:
kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=worker1

参考：

Multi-node Kubernetes setup on Windows – Technology evangelist

https://medium.com/@srpillai/single-node-kubernetes-on-centos-c8c3507e3e65

https://www.cnblogs.com/wangguishe/p/17582668.html