本地用虚拟机搭建 K8S 集群

本地用虚拟机搭建 K8S 集群

一、安装配置 Ubuntu 虚拟机

如非特殊说明，以下操作均在 root 用户下

1.1 安装系统

用 VMware fusion 安装一台 20.04.3 的 Ubuntu 虚拟机。IP 先自动分配，安装时选装 openssh。

1.2 固定 IP 地址

# 安装 net-tools
apt install net-tools
# 执行 ifconfig 查看 gateway 地址， 以及默认分配的 IP 地址
vi /etc/netplan/00-installer-config.yaml
# 配置文件如下， addresses 和 gateway4 需要修改
# 使配置生效
netplan apply

network:
 ethernets:
   ens33:
     dhcp4: no
     addresses: [172.16.85.133/24]
     optional: true
     gateway4: 172.16.85.2
     nameservers:
            addresses: [223.5.5.5,223.6.6.6]
 version: 2

1.3 修改系统配置

# 关闭 swap
swapoff -a
# 最后一行 swap 的注释
vi /etc/fstab
# 配置时区时间
timedatectl set-timezone Asia/Shangha
systemctl restart rsyslog
systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

# 修改其它参数
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sed -i 's/2/1/g' /etc/sysctl.d/10-network-security.conf

# 使系统配置生效
sysctl --system

二、安装软件

2.1 安装 docker

mkdir -p /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{ 
 "registry-mirrors": ["https://7liizqnc.mirror.aliyuncs.com"], 
 "live-restore": true , 
 "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
apt install -y docker.io

2.2 安装 k8s

apt-get update && sudo apt-get install -y ca-certificates curl software-properties-common apt-transport-https curl

curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

tee /etc/apt/sources.list.d/kubernetes.list <<EOF
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

sudo apt-get update

apt install --allow-downgrades --reinstall -y kubelet=1.18.20-00 kubeadm=1.18.20-00 kubectl=1.18.20-00

三、复制虚拟机

3.1 虚拟机关机

关机后，直接复制虚拟机文件复制两份。

3.2 修改 hostname 和 IP

挨个启动，然后修改 hostname 和 IP

# 修改 hostname 
vim /etc/hostname
vim /etc/hosts

# 修改 IP 地址，固定 IP，只需要把 /etc/netplan/00-installer-config.yaml 中的 addresses 修改，然后 执行 netplan apply

四、配置 k8s Master

启动 master 虚拟机

4.1 修改 kubelet 配置

# 执行 systemctl status kubelet ，查看 kubelet 服务的配置文件地址。 一般是 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

# /etc/systemd/system/kubelet.service.d/10-kubeadm.conf 配置如下，其中 Environment 最后增加了 --cgroup-driver=systemd

# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=systemd"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

# 然后加载配置
sudo systemctl daemon-reload

4.2 初始化

kubeadm init --pod-network-cidr=172.244.0.0/16 --apiserver-advertise-address=172.16.85.133 --kubernetes-version=v1.18.20 --ignore-preflight-errors=Swap --image-repository registry.aliyuncs.com/google_containers

# 注意其中 --apiserver-advertise-address 要修改为 master 的 IP 地址，也就是本机的。

执行完成后会有如下输出

[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.16.85.131:6443 --token 94z9t6.hm9d72week7862em \
    --discovery-token-ca-cert-hash sha256:585f9830fc220b22ada827b710df83cc3a86037fbd38f128798fb2d91a338629

如果想要在普通用户中执行kubectl 命令操作集群，则需要在该用户下执行如下命令：

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

4.3 初始化网络模块

kubectl apply -f https://docs.projectcalico.org/v3.20/manifests/calico.yaml
#执行完成后执行 kubectl get nodes 查看状态，直到 status 为 ready 则成功。

4.4 安装 dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml

# 执行如下命令查看是否就绪， running 就绪
kubectl -n kubernetes-dashboard get pods

# 就绪后设置浏览器访问, 浏览器访问的端口是 nodePort, 注意协议是 https
kubectl patch svc kubernetes-dashboard -n kubernetes-dashboard -p '{"spec":{"type":"NodePort","ports":[{"port":443,"targetPort":8443,"nodePort":30443}]}}'


# 创建 dashboard 的用户
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard

# 授权
kubectl create clusterrolebinding dashboard-admin-rb --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin

# 获取用户的 token，浏览器访问 dashboard 需要输入 token
kubectl describe secrets $(kubectl get secrets -n kubernetes-dashboard |grep dashboard-admin | awk '{print $1}') -n kubernetes-dashboard

五、加入 node 节点

# 将 master节点的/etc/kubernetes/admin.conf 复制到 node 节点相同位置

# 在 node 节点执行
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# 加入 k8s 集群， 这个命令是 master 初始化结尾的命令。
kubeadm join 172.16.85.131:6443 --token 94z9t6.hm9d72week7862em \
    --discovery-token-ca-cert-hash sha256:585f9830fc220b22ada827b710df83cc3a86037fbd38f128798fb2d91a338629

# 查看是否加入
kubectl get nodes